pfh - Network

https://logarithmic.net/pfh-files/random/grad.mov

https://logarithmic.net/pfh-files/random/grad.mov

Happy little accidents...

Happy little accidents...

I'm really liking this course on generative diffusion models. They seem to have boiled many years of confusing development of ideas down...

I'm really liking this course on generative diffusion models. They seem to have boiled many years of confusing development of ideas down to a simple approach.

https://diffusion.csail.mit.edu/

The optimizer tries to find the lowest energy. This closely resembles "maximum likelihood" or "maximum a posteriori"...

The optimizer tries to find the lowest energy. This closely resembles "maximum likelihood" or "maximum a posteriori" estimation in statistics. We might hope this finds the most representative estimate. Clearly, here it does not! The estimate is smoother than most samples from the distribution.

Also the optimizer has not found the lowest energy state, which would be all-dark or all-light. It might take a very long time to reach one of these optima!

Second, sampling from the distribution with a Langevin Dynamics simulation. The algorithm is almost identical to gradient descent with...

Second, sampling from the distribution with a Langevin Dynamics simulation. The algorithm is almost identical to gradient descent with momentum, but we add just the right amount of noise to the momentum at each step.

Comparison of optimization and sampling from a distribution defined by an energy function. I use a continuous version of the Ising model...

Comparison of optimization and sampling from a distribution defined by an energy function. I use a continuous version of the Ising model spin lattice energy.

First, optimization from a random initial state using gradient descent with momentum, using the SGD optimizer in PyTorch.

A common pattern for me is a "chunked flow" (better name desperately needed). R very much leans towards operations on complete...

A common pattern for me is a "chunked flow" (better name desperately needed).

R very much leans towards operations on complete datasets. A flow of data feels more Python-ish than R-ish. Working with chunks in R retains most of the efficiency.

In a local({ }) block I have one or more data generators producing chunks, which undergo processing and then are saved in one or more outputs (often parquet files).

A further refinement is to farm out processing of the chunks with multiprocessing.

Also, each "with_" function has a corresponding "local_" function that stacks into your current environment (a function...

Also, each "with_" function has a corresponding "local_" function that stacks into your current environment (a function body or local({ ... })).

This is syntactic sugar to flatten nested resource usage, much like the pipe %>% flattens nested function calls. Makes coding easier for humans.

I think I'm a little bit in love with the {{withr}} package in R.Similar to "with" in Python, you can guarantee to properly...

I think I'm a little bit in love with the {{withr}} package in R.

Similar to "with" in Python, you can guarantee to properly clean up when using a resource such as a connection to a file, a temporary directory, or a temporary global setting change.

https://withr.r-lib.org/

Some reflections on the series here.https://logarithmic.net/pfh/blog/01735193695

Some reflections on the series here.

https://logarithmic.net/pfh/blog/01735193695

I've been watching the 2023 Statistical Rethinking lecture series by Richard McElreath. These cover a complete approach to statistics...

I've been watching the 2023 Statistical Rethinking lecture series by Richard McElreath. These cover a complete approach to statistics based on causal reasoning and Bayesian analysis. They are excellent, highly recommend.

https://www.youtube.com/playlist?list=PLDcUM9US4XdPz-KxHM4XHt7uUVGWWVSus

Ideas I've stopped believing in: effective altruism.This is a utilitarian approach to altruism. The most good can be done by earning as...

Ideas I've stopped believing in: effective altruism.

This is a utilitarian approach to altruism. The most good can be done by earning as much money as possible then donating it. For example Sam Bankman-Fried attempted this. He didn't get to the donating step, but in principle it could have been a good gamble that didn't pay off. Next we think about the long term, the most good for future generations, including being born at all. Finally we imagine those future generations being a certain race.

The plural of anecdote is anecdotes.

The plural of anecdote is anecdotes.

https://pfh.github.io/rezbaz/myplot.html

https://pfh.github.io/rezbaz/myplot.html

I just ran my 1.5 hour HTML+SVG+JavaScript+D3 workshop at ResBaz Victoria 2024. Touched on a minimal set of ideas ideas needed for an...

I just ran my 1.5 hour HTML+SVG+JavaScript+D3 workshop at ResBaz Victoria 2024. Touched on a minimal set of ideas ideas needed for an interactive web-page data visualization, and even uploaded to github pages. #resbaz

https://pfh.github.io/jsfoot/

So I'm thinking it might be possible to take a policy like BH which is maybe not quite right in a particular setup and *recalibrate* it...

So I'm thinking it might be possible to take a policy like BH which is maybe not quite right in a particular setup and *recalibrate* it based on simulation or a resampling scheme. For example a Tukey all-pairs comparison version of FDR, where the test statistics aren't independent. Or gene-set enrichment.

Pondering FDR control. Procedures like BH represent a certain policy, which can be confirmed to work by simulation for any given number of...

Pondering FDR control. Procedures like BH represent a certain policy, which can be confirmed to work by simulation for any given number of true discoveries. The actually achieved FDR guarantee is the FDR obtained in the worst case.

Something that should exist:geom_principal_curve()If x and y have a symmetric relationship, eg both x and y are noisy measurements of an...

Something that should exist:

geom_principal_curve()

If x and y have a symmetric relationship, eg both x and y are noisy measurements of an underlying hidden variable, geom_smooth will under-estimate the slope, just like linear regression.

Prompted by a case where the slope really should have been 1. geom_smooth made it look less, and even made it seem like the data should be broken into groups.

⟋
⟋
⟋

sgdGMF is a general purpose matrix factorization package. Like PCA, but better, e.g. works well with count data. Should be fast, e.g....

sgdGMF is a general purpose matrix factorization package. Like PCA, but better, e.g. works well with count data. Should be fast, e.g. suitable for scRNA-Seq. Presented by @drisso at #abacbs2024.

https://github.com/CristianCastiglione/sgdGMF

The Knockoff Framework looks very interesting as a broadly applicable method of maintaining an FDR. (Guannan Yang talked about this at...

The Knockoff Framework looks very interesting as a broadly applicable method of maintaining an FDR. (Guannan Yang talked about this at #abacbs2024)

https://web.stanford.edu/group/candes/knockoffs/index.html