# Understanding and Implementing the hypergeometric test in python

Whenever I find a topic I can’t find a sufficiently good tutorial or explanation of online, I feel compelled to offer one. I hope this helps you.

## I. Understanding the Hypergeometric Distribution

The hypergeometric distribution describes the probability of events in the following scenario:

Suppose you have a jar containing 10 red marbles and 90 black marbles.
You collect 10 marbles from the jar.
What is the probability you collect k red marbles?

Collecting a single red marble seems intuitively most likely, but if you collected none or a couple, that wouldn’t be too surprising. But what if you collected 6, or 8? …

# Logistic Regression

With binary logistic regression, the goal is to find a way to separate your two classes. There are a number of ways of visualizing this.

No matter which of these you choose to think of, we can agree logistic regression defines a decision rule

h(x|theta) = sigmoid(x dot theta + b)

and seeks a theta which minimizes some objective function, usually

loss(theta)= ∑ y*log(h(x|theta)) + (1−y)log(1−h(x|theta))

which is obfuscated by a couple clever tricks. It is derived from the intuitive objective…

# Models

“ ‘All models are wrong, but some are useful.’

So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if imperfectly, explain the world around us. Until now. Today companies like Google, which have grown up in an era of massively abundant data, don’t have to settle for wrong models. Indeed, they don’t have to settle for models at all.”

So proclaimed WIRED editor-in-chief Chris Anderson 7 years ago, opening the July 2008 issue of stories relating to the advent of “The Petabyte Age” with his piece entitled: “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete”. …

# A note to future hackathon organizers

## Reflections from Tufts Polyhack, October 23–24, 2015

Some four years ago, I attended Tufts’ first hackathon put on by Marshall, and it changed the course of my life in a pretty profound way. Seth Drew, Teddy Cleveland, Alex Daniels and I built an app we called RaptorScheme, which was some handwritten html, css, a violently altered D3 example we mutated for our purposes, and some python scraping to get course data.

It was really bad, but it was also shockingly good given that we had been writing code for a timescale of weeks. I remember how, knowing no javascript, let alone D3, Seth and I spent the wee hours of the morning decyphering each of the 300 lines of D3 example code vaguely related to what we wanted to do. We read each line out loud and pondered what it could mean, argued about what was going on, and finally, triumphantly, developed a sufficiently accurate mental model of how it all worked to alter it for our purposes. …

# Led By Sheep

Climbing rocks is probably the best way I know of to slowly lose all your time and money. There is hardly a better way of doing so.

I was climbing near Aspen, CO, when the opportunity arose to go to southern Utah to see my godfather’s daughter (also my father’s goddaughter) who was spending a year abroad at Southern Utah University (she’s French). My brother and I naturally leapt on the opportunity to explore another part of the climbing world: Zion National Park.

So we picked up the SuperTopo book for Zion. ProTip for new climbers: SuperTopo & Chris McNamara are the very best thing that has ever happened. We accidentally own most of the books now. …

# That Time We Built a Climbing Wall In Our Backyard…

For me, Climbing is yoga on a vertical surface. It’s a negotiation between your will and gravity, and demands a holistic self-discipline I had been craving without knowing it. And its unfathomably addictive.

I started rock-climbing on a whim. Sometime in early October of last year, a friend invited me to the rock gym one morning. By my third trip to the gym, I was absolutely enthralled.

Around the same time, I found out that just about everyone I knew either was already a climber or had picked it up at the same time I had, including my little brother Max. That winter, the day after I returned from Patagonia, January 7, I went into a long-delayed shoulder surgery. I woke up the next day and understood that the time I had budgeted to “recovery” was going to be very monotonous if I didn’t find a way to fill it. I had three weeks to kill before school started back up. I brainstormed projects with Max, and one of the first ideas he had was that of building a climbing wall in our backyard. …

# I want to send emails from my google domains email through gmail.

I registered my domain foo.com through google domains, and then I setup email forwarding so that emails sent to alex@foo.com go to alex@gmail.com, but now I want to send email from alex@foo.com.

Well, I went through the support channels at Google domains — so you don’t have to.

To set up your Gmail to send using an alias, 2-Step Verification must first be enabled (you can disable it right afterwards). To enable 2-Step Verification, follow the instructions on this link. Once enabled, follow these steps:

2. In Gmail, click your username or user icon in the upper right corner to bring up the user menu. …

# The way we talk about Ethics is broken

## An analogy with the Neural Network Algorithm

I’m going to try and explain to you why I believe discussions on the subject of ethics, especially of the abstract variety, in which seemingly well warranted postulations about the nature of morality abound, are not only entirely futile but morally hazardous. But in order to do this, I need to explain to you how I arrived at this conclusion, and that’s going to require that I provide you some background in Machine Learning…

The Artificial Intelligence community is abuzz about a buzzword which has begun to permeate the public psyche: Deep Learning, the return of Neural Networks, just bigger.

Neural Networks are a category of machine learning algorithm originally inspired by the brain. They abstract and formalize ‘neurons’ as units which send a signal to the next layer of neurons if the combined strength of the signals received by that neuron (via metaphorical dendrites) exceed some threshold. …

# Torres Del Paine ‘O Loop’ in 4.5 days

## But is that enough time to fully savor the park?

Torres Del Paine is one of the great natural treasures on the face of this planet, and a friend and I recently had the great pleasure of experiencing it. The Torres were the first stop in our Patagonian expedition, which was all too short, and wanting to do as much of Patagonia as possible in the two and a half weeks that we had, we opted to try and do the famous ‘O-loop’ as quickly as it could be reasonably accomplished while still enjoying the trekking and scenery. Turns out this was roughly 4.5 days.

# Optimizing Transmission of Knowledge

In response to The Economist’s The Future of the Book

To me, the really fascinating thing about books is the ecosystem they've given rise to. Books have essentially promulgated a set of cultural artifacts and institutions which allow them to replicate and disseminate themselves safely throughout our world. As Richard Dawkins writes in ‘The Selfish Gene’, information has an extraordinary capacity to simultaneously insulate itself from but also radically alter the physical world, and as vehicles of information, books fit neatly within that idiom.

Books have dedicated machines to their production, an economy around their invention (with people paid to naturally select the most fit for wide reproduction) and a variety of places dedicated to them, notably libraries and book stores. Books even have a piece of furniture especially for them. But perhaps most notably, books have established themselves as the primary symbols of passion for the advancement of humankind, garnering themselves a number of ardent defenders dedicated to their survival: scholars and librarians alike fight to preserve and protect them. …