real webdev

Honestly, we're needing to look at groupings of information knowledge. It's not just masters in statistics and math and database development, it's connecting these to an interface (web, phone), and then securing them.

We can also look at our own life in terms of a database, the way various government agencies probably do. If we view ourselves and other citizens as entities, attributes, and relationships. In the case of events or incidents, each one one of these might be an entity, eg a car accident, which is a singular event.

formal

Masters in statistics, math, computer science, just to get into other classes.

Is discrete math the intersection of statistics and maths?

database

terminologies, relational, normalization,

database normalization (28:34) Decomplexify, 2022. simple examples and repairs on 1-5nf in RDBMS. columns (attributes), rows (entities). Design normalization can protect against things that cannot be logically true like two dates of birth. Eliminates data that disagrees with itself, and as a side benefit, redundant data.10:20 2nf deletion, update & insertion anomalies. 3nf 17:30 transitive dependency. 19:00 golden rule of boyce-codd.
relational terminology (8:48)Neso Academy, 2022. terminlogy for RDBMS.

sets of information

cobol, php (and whatever replaces it -- can it be done with python?)

statistics -- some foundation

Statistics overlaps into Math of course. We can record samples into a CSV or spreadsheet and do histograms, and (Algebra 1) regressions. Yet CSV data are arrays. Arrays are matrices, and thus vector, evaluable (Calc 3), as well as Python and R friendly. IMO, because it spans both simple and complex Math, and CS, Stats are a fun side project during one's Math progression, at any time.

Basics are Algebraic if one uses tables for area under curves. If not, a person could do them using simple Calc 1 calculations. TI-84's -- if available -- are also great for checking that kind of calculation (eliminating tables and Calc) and for offloading the drudgery of long lists.

HS AP Stats is essentially an Algebra-only college Stats1 class, and slightly more user friendly. A review book for AP Stats, the Princeton Review, Table of Contents is a list of roughly 35 subjects, split over 4 categories. I've used it below with some modifications to describe what a person might want. Beyond basic or AP, I've added a "next level" section at the bottom, since matrices connect Math to Statistics and arrays/data structures connect Stats to Data Science.

Introductory data data collection tabular methods graphical methods - qualitative graphical methods -quantitative numerical methods - continuous boxplots add/mult a constant comparing mult. groups bivariate data: covar,regression categorical frequency Sampling and experiments plan a study data collection plan a survey bias in surveys plan an experiment Anticipating outcomes probability

random variables probability dist. - discrete random var's probability dist. - continuous random var's normal distribution combining independent random var's sampling distributions Statistical inference confirming models parameters point estimation interval estimation confidence interval inference: significance tests hypothesis: testing and accepting estimation and inference: population proportion estimation and inference: population mean estimation and inference: 2 population proportions estimation and inference: 2 population means inference: categorical data

For a comprehensive course, there's Khan Academy. On YouTube, I prefer Brandon Foltz, M.Ed video compilation, esp at 0.5x speed. They were made 2010-ish, but ahead of their YouTube time¹. Below there are some from Foltz and from many great teachers or vids which include nuggets in some way. As one goes along in review, it's a chance to relearn the annoying Greek symbols for parameters. The same Greeks as in finance, but different usage.

¹Another brilliant info-sharing hero with early YT chops was Derek Banas. One of his best might be his investing video, which crosses data science with financials and Python. And of course Barry Brown for programming any of it in C.

---

distributions of data obtained

degrees of freedom, clt

To Z or t (38:16) Brandon Foltz, 2012. The notion behind this choice, without calculation. Almost all stats are samples, not entire population, ie, "the prevalence of depression of those over 65" - how to give all questionnaire? Can't. The smaller the number, the more chance of error. Avg temp in NYC, what if we just used a day, or 5 days? The larger the sample the more confidence we have captured reality.16:00, 33:00 decision. 19:00,30:00 degrees of freedom
Buying land (32:06) Brantley Blended, 2018. listing, PLAT recency, survey recency (due diligence period), boundaries,

bivariate data: linear regressions, covar

covariance (5:55) Ben Lambert, 2013. intuition behind covariance. Positive, negative, and none. Some formulas, but conceptual.
Variance vs. covariance (Webpage) Investopedia.

combinatorics, permutations, probability

nCr and nPr. These are the denominators of probability, since they give the universe by which we determine our odds. If "statistics" are the collected data, nested inside we have probability (based on prior events, statistics), and inside this is the "combinatorics and permuatations" which create the denominators of probability fractions.

Permutations and Combinations (17:40) Organic Chemistry Tutor, 2017. Simple review of when to use nCr, nPr, or just the factorial by itself. 14:00 How many ways can we arrange the letters in the word "Alabama".
problem - probablility (replace/don't replace) (10:11) Amy Krusemark, 2020. a slight political note at start, AP question. example of non-replacement probability, example of understanding permutation/combinatorics effecting denominator. 2:40 without other notice, 0.05 alpha is threshold for a reason to doubt.

---

next level

KL Divergence

KL Divergence (18:13) ritvikmath, 2023. Non-negative comparison of two distributions.

vectors and matrices in statistics

As my friend Bart texted...

Matrices are really just notation for a list [of] equations. Not so profound, not for a long time. Like if I have ten data points then i have that equation ten times. Ten y values. Ten x1 values, ten x2 values, etc. Thirty diff x values, so x could be written as a 10x3 grid. Ok enough [of] that
Beta is 1x3 and and x is 3x10. Then when multiplied out gives a 1x10 vector. To say that vector equals the 1x10 y vector is to say each component of y equals the corresponding for RHS. ie ten equations

And of course wherever we have a matrix, we have a potential vector.

Comp Sci v. Data Sci Matrix (10:34) ritvikmath, 2019. Reveals some clear differences between computer matrix use and math/datasci use and how they overlap (eg. CompSci efficiency can work on any math app).

data structure

necessary for data science. Data Science will evaluate these further, sometimes using Calculus, but we at least need to know what they are, IMO. Not on the AP test. The terminology transform is Statistics and Math use the term "matrix", but Data science/Computer science use the term "array". Depending on what we need to do with the matrix, computing will perform some function on a data array. Usage: imagine an R2 scatter plot, but where we have a third dimension with error information attached to each data point.

Linear Algebra: Transformational Matrices Part I (15:43) Computer Science, 2021. transformations on R2 matrices, using geometric examples. There's an entire playlist that's valuable.
Linear Algebra: Transformational Matrices Part II (9:23) Computer Science , 2021. transformations on R3 matrices, as we might do with data structures.
Trig Functions (9:15) PatrickJMT, 2011. this hero scores yet again. Just in case you need them again for the stuff above. So rotten.
1,2, and 3d structures (8:32) GridoWit, 2017. basic terminology and location tracking within different types of arrays and data structures. The inuition that arrays solve a storage problem: we don't want to have a new variable for each piece of data we have collected. C syntax is also provided.

statistics -- simplistic error catching in studies

Note: I've also included some statistics-relevant information from my 2020 Maths post within a "resources" section at the bottom of this post.

This list so far only includes traditional "p-value" errors, and not Bayesian style errors. But when we look at published papers in Nature or at the NCIB, what are some rule-of-thumb errors we can look for?

short list

• correct test for the hypothesis
  Look for a hypothesis to be clearly stated and that the null has been rejected. That said, having pre-determined hypothesis and the requirement for rejection of null is an outdated method to some: a more modern approach is to crunch an immense amount of data through arrays and let the data itself identify anomalies from which to draw conclusions. However, this method is still used in many studies required by, eg. the FDA and many scientific papers.

  Types of tests (19:53) Amour Learning, 2019. Generally t tests make less presumptions than Z tests and are better. 12 main types of tests and when to use for which hypothesis.
  Selecting the correct test (12:31) Erich Goldstein, 2015. What type of data (categorical or quantitative)

• a priori standards
  Is the alpha value determined prior to the study and clearly stated? A typical alpha value of 0.05, looks for a 1 in 20 chance the results would happen by chance, and obviously an alpha value of 0.10 that 1 in 10. NCBI:nih.gov
  Counter: the most common reduction is for a third variable. I'd like to propose another: studies designed with an a priori alpha presuppose the a priori hypothesis on which is reliable. However, data gathered in the study may reveal other information. There may be some statistical significance for although it's not clinical.
• publication bias
  My friend Bart notes

  If ten people examine a question and none of them do any datamining and only the people who happen to get a positive result submit or publish, then the stats seem ten times as strong as they are.

• randomization problems
  Front-end randomization is a thing, but there are other error possibilities later in the study also, well-described here.
• control group
  Perhaps too obvious to include, but included anyway

resources

p value/alpha value

Only Bayesian statistics can prove theories, but P value stuff can prove data. It's a lower standard than Bayesian but useful, of course.

What's a P value (20:30) Daniel Lakens, 2019. some accent -- good at 0.75x. Good backgrounder. No math. Notes Bayesian necessary on theories. 8:45, otherwise all we can do is rule out something (using P).
Baye's theorem (15:10) 3Blue1Brown, 2019. new information, and how to use in research. Graphically illustrated.

12 main types of hypothesis tests

As noted above, they must be matched to the hypothesis, but what are they?

concept instruction

The clearest statistics instruction videos for my time are from Brandon Foltz. "Only" an M.Ed (probably now a Phd), found ways to create clean graphics, and years before clean graphics were common. Another excellent early adopter, although only in an overlapping sense with statistics, is Derek Banas. One of his best might be his more recent comprehensive portolio using Python. I believe Derek had a job with Apple when he was 16.

spreadsheets, Python, R, SPSS

Most people cannot afford SPSS, ergo PSPP is a a good option. If using Linux, most repo's have it. In Arch, it's in the AUR. There's plenty of PSPP videos on YT, as well as SPSS videos that a person can adapt.

This guy's series is *incalculably* helpful, har har. He as MSoft AF, Vstudio, Excel, Jupyter Notebook, but it's just as easy to do all of his stuff in Google Colab with Google Sheets as-is, or download them as CSV's first and then merge them.

Pandas Merge (Excel) (9:08) Alan Hettinger, 2022. Similar to a SQL join. When we want to put things together from various tables into a combined output.
Pandas Merge (Sheets) (10:48) Tobias Willman, 2020. Adapt the Jupyter above using this guy's vid.

maths resources

Links: Photomath :: Wolfram Alpha :: Desmos
xda xiaomi redmi2 :: xiaomi community :: more flashboot commands :: reddit post (self-teach calc) :: Ranking engineering degrees by difficulty (15:07), 2023

---

Math is difficult, because the explanations come too late in the game. You have to get to Linear Algebra to explain basics, which most people can't accomplish

spreadsheets, Python, R, SPSS

Most people cannot afford SPSS, ergo PSPP is a a good option. If using Linux, most repo's have it. In Arch, it's in the AUR. There's plenty of PSPP videos on YT, as well as SPSS videos that a person can adapt.

Python, Numpy, Plotly (1:10:57) Derek Banas, 2022. Uses Jupyter on Anaconda but a couple tweaks will do it in Google Cloud. 16:00 explains Numpy, 17:00 read CSV, scraping 21:00, Plotly 45:00
Pandas Merge (Excel) (9:08) Alan Hettinger, 2022. Similar to a SQL join. When we want to put things together from various tables into a combined output. Can dapt Jupyter to Google Colab with Google Sheets as-is, or download them as CSV's first and then merge.
Pandas Merge (Sheets) (10:48) Tobias Willman, 2020. Adapt the Jupyter above using this guy's vid.

Self-study generally

self-study feynman's books (27:33) Math Sorcerer, 2023. Mathematics for self-study series of books, espec trig and calculus.
V600 scanning tips (5:53) erwnalejo, 2021. Somewhere in Western Kansas. Lomo digitaliza, blue tack to adjust height. Links in comments to products. Lomo is an extra $50.
epson scanning tips (28:11) Nick Carver, 2019. Paper masks some of the holder. 5:40 uses a squeeze ball. 2400 dpi is fine for any normal use including 8.5 x 11 blowups, etc.
V600 overview (52:17) Film Friends,

PreCalc

Two best books I know are Larson and Sullivan's. I found a 9th edition Sullivan and a

Polar coordinates

Difficult because we move back and forth between analytic trig and graphic trig, then introduce time changes in the angles

Series

Infinite series means infinite terms, not that the sum diverges to infinity. Also the best way to learn them is M1 in MVP math.

Harmonic Series (sum of sums) (46:34) Mathologer, 2020. Infinite sum of 1/(n+1) Gamma explained at 24:21.
Buying land (32:06) Brantley Blended, 2018. listing, PLAT recency, survey recency (due diligence period), boundaries,

Matrices

My road to matrices has been difficult, b/c for years it was one of those categories where 1) I simply had to remember formulas and 2) Math teachers acted like matrices were easy, like I was an idiot, because the problems were easy. Nearly always in Math, my intuition is right when something doesn't quite seem that easy. So lets call matrices what they are: representations of vectors with several degrees of freedom, not just 'simple arrays'. Although the latter is correct, it's like saying cars are easy to make b/c they fit in a garage.

The real way to look at it is via transformations, rotations, and vectors, as well as ways to write functions (remember we can also write functions with limits). That is to say, linear algebra. M

1. Berkeley HS Math 1, Module 6 (webpage). This has some good videos on different transformations: reflections, rotations, and translations. The shortcoming is that the they move the points, instead of changing the underlying grid (as we do with matrix scalars).
2. Linear transformations (10:58) 3 Blue 1 Brown, 2016. The real value of what matrices actually are, and also acknowledged by the host. They know how important this is. Connecting matrices and vectors. 3:52 i-hat (x), j-hat (y).
3. Why can't we multiply vectors (51:15) Freya Holmer. A good review, awkwardly stated, of basic maths up to matrices/vectors. Multiplying R3 vectors gives quaternions, multiplying R2 vectors gives us complex numbers.
A tour of differential equations(27:16) 3 Blue, 1 Brown, 2017. How we work backwards with diff equations
4. interesting vector usage (14:40) Kieran Borovic, 2023. an interesting way to convert categories of nature into a 7 row vector.

Integral tricks

Most difficult are problems that combine U-sub (reverse chain) and IBP, integration by parts, but also must know d/dx, dx/dy, Dx

Integral conceptualization (10:54) Math The World, 2024. *does not* use the best Cherkassky/Russian way of series, however gets much nearer than most and points out the problem of focusing too much on area under curve.
Why dx at end of integral (4:36) Krista King, 2016. sometimes I forget why it's smashed onto the end of an integral or double integral and so this is a good reminder it's the dx(limit of distance,0), not the Dx (distance).
What is dx alone? (5:38) Jim Fowler, (2013). Just as important as the one above. A little deeper.
u-subs (11:02) The Organic Chemistry Tutor, 2018 . a few definite integrals from organic chemistry tutor.
Cheat sheet (PDF) Most of the tricks and common integrals.

Exponents/Logs

exponentiate (30:08) Mathemaniac, 2024.

Lesser used

I consider that these are like the irregular verbs of math. We have to know them typically by use, as they are difficult to remember how to derive. Trig identities, hyperbolic derivatives and integrals. Geometric Algebra -- nuclear physics.

Logic

Russell's paradox (28:27) Jeffrey Kaplan, 2022. Covers naive set theory inadvertently (as opposed to axiomatic set theory).
attach/remove ft lens (2:46) Caleb Ginsburg, 2021. Turn to f5.6, then turn speed selector to 400 ASA in alignment with 5.6, then push the release button. It sort of hinges from top to bottom.

Statistics

Old skool brandon foltz, are still some of the best videos out there.

the most important statistical skill (13:35) Very Normal, 2024. Monte carlo is about iterations