nikhil.io

Plurals in Python

>>> n = 0
>>> print "%d item%s" % (n, "s"[n==1:])
0 items
>>> n = 1
>>> print "%d item%s" % (n, "s"[n==1:])
1 item
>>> n = 2
>>> print "%d item%s" % (n, "s"[n==1:])
2 items

# If you might want to print negative items, add abs to the test:
>>> n = -1
>>> print "%d item%s" % (n, "s"[abs(n)==1:])
2 items

# If a word has irregular plural morphology, use a list:
>>> n=1
>>> print "%d %s" % (n, ['abacus','abaci'][n!=1])
1 abacus
>>> n=2
>>> print "%d %s" % (n, ['abacus','abaci'][n!=1])
2 abaci

Source.

What David Foster Wallace Circled in His Dictionary

by Slate Magazine and the Harry Ransom Center

“Calling Bullshit” is a free online course that helps you “detect and defuse” Bullshit when you encounter it.

Our learning objectives are straightforward. After taking the course, you should be able to:

  • Remain vigilant for bullshit contaminating your information diet.
  • Recognize said bullshit whenever and wherever you encounter it.
  • Figure out for yourself precisely why a particular bit of bullshit is bullshit.
  • Provide a statistician or fellow scientist with a technical explanation of why a claim is bullshit.
  • Provide your crystals-and-homeopathy aunt or casually racist uncle with an accessible and persuasive explanation of why a claim is bullshit.

Further,

We will be astonished if these skills do not turn out to be among the most useful and most broadly applicable of those that you acquire during the course of your college education.

I don’t think there’s more critical a juncture than now for courses like these. Just wish more people took them.

Ayn Rand the Philosopher

by samiiRedditBot

“Two novels can change a bookish fourteen-year old’s life: The Lord of the Rings and Atlas Shrugged. One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other involves orcs.” – Kung Fu Monkey

Ayn Rand is to philosophy what Twilight is to horror fiction - being that the only people who like it have absolutely no experience of what the subject is really about and as a result have nothing to compare it to. Her influence only extends to the general public and she is pretty much ignored by academia, and even then she is more or less unknown outside of North America. I suppose that if you’re a fan you might think this is due to her ideas being so radical that they threaten the liberal elite of University Academia, who plot to turn the USA into a communist country, or whatever. However the reality is that it’s just really bad to the point of being actually quite painful to read if you know anything at all about real philosophy. Reading the woman is to philosophiers like watching CSI if you know anything about computers - just painful.

What Ayn Rand supposed is that there exists a absolute reality of the A=A. Essentially in her mind existence exists and everything that man does is merely the reordering of existence. She goes on to argue that things like art enable man to create metaphors in reality enabling their wildest metaphysical dreams to take flight. She’ll argue that things such as visual illusions, do not prove that that man can not trust their perceptions but rather their perceptions are always correct and any mistakes are mistakes of interpretation, rather than perception of reality. She argues that since you are able to draw absolute logical conclusions and concepts from reality, that the only valid form of logic is Aristotelian logic because it is the one most grounded in reality.

She argues that the ideal world is one where all men are following their own “rational self interest” that they have drawn from reality, which in her mind is represented via a perfect capitalist society, and that there exists a “conspiracy” to suppress men by making them “unsure” of reality. She’ll redefine terms like selfishness and altruistic towards this end. For example a moral individual is one who follows their own “selfish” interest to build houses or whatever but that this is always undermined by “altruistic parasites” who will loot the efforts these selfish individuals rather than going out and doing things for themselves, mainly because they envy the selfish.

Now, as far as philosophical views go this is a pretty loony view of the world. Here are some objections

Her views are over simplistic solutions to complex problems, for example even if you suppose that there exists an absolute reality then you’re still left with the problem of how you, as a subjective individual, reconcile your abstract ideas to it - these are the sort of problems that real philosophers attempts to address. Basically in saying “existence exists” you haven’t actually “solved” anything but merely misdirected attention away from the problem. It is like saying that to end war people should just stop fighting, yeah sure, that people would stop fighting might be a consequence but then how would people otherwise solve disputes? It is childish logic, essentially.

Her views are dogmatic, that is they are true because you believe them to be true rather than being able to prove it to be true. There is this attitude that no one shall enter the kingdom but through me, this is not the way philosohpy works - because philosophy is dialectical - but rather the way religion works. What she is doing in saying that reality objectively exists is to make her personal subjective views appear to be more concrete than they actually are. Which is what her true intent is: to turn her specific definition of what capitalism is into some weird religion for the gullible, and nothing more.

She simply makes stuff up, for example she supposes that the whole of 19th century Idealism is merely a conspiracy to make people unsure of their own existence so that they are easier to control by 'altruistic ’ dictators. She spends the vast majority of her time in her books simply constructing and then knocking down various strawmen, who are represented by her characters. For example Dr. Floyd Ferris.

Her work is just derivative of other better philosophiers such as Nietzsche, that she has simply rewrittern a bad interpretation of in order to sell her rubbish moral opinions. As a consequence you’re much better off just going to the source and cutting out the middle hag, espically since she seems to have a rather limited grasp of genuine philosophy, and much of anything in general. This is very apparent if you even have a basic working knowledge of philosophy.

She’ll just ignore her own arguments when they turn out to be inconvient. For example she’ll celebrate the triumph of the Apollo project which being a government funded collective effort is the very antithesis of everything she is about. Brushing it off with broad statments like: “It would have been done” etc. And everything about who she was, how she became who she was and how she lived her life point towards a high level of cognitive dissonance. In fact there are convenient loopholes in her philosophy that state that her arguments can be ignored based a emergency situation, such as being on a life boat etc.

But the greatest failing is that even if you subscribe to her “philosophy” and ignore the above, even then it still falls down by its own argument. For example suppose I give you $5 for a hotdog, now as a objectivist to suppose that some benefit has occured between both parties then you have to presume that money is actually worth something outside of a social context, otherwise nothing has been created in the transaction since at the end of the day all money is, is a abstract concept. To put this another way, if I gave you $5 for a hot dog then there would be absolutely no benefit to you if you are unable to spend it due to it not being legal tender. To quote Adam Smith: “All money is a matter of belief”, currency is only currency because of social and economic forces and not because it is instrinically worth anything. It is not linked to reality in anyway, at all, but is only symbolically representative of reality. Do you see how it defeats itself? if objectivism is followed to it’s logical conclusion then money is worthless, in a supposed capitalist utopia

In conclusion: Rand just wrote a very bad interpretation of Nietzsche for young adults. Where Nietzsche argued that the problem with society was that it was based in ideals, E.G., Christianity, that were essentially founded in nihilism and were therefore self-destructive because they were effectively internalizing resentment and suppressing human passions, and went on to argue that the cure to this was to be found in the “free spirits” being those able to break free from this endless pattern of idealized self-destruction by getting back in touch with their own human drives, who love fate, who are (or do not resent) the strong and who reject notions of absolute ideal truth. What Rand does is take this view literary and argue that rather than rejecting it you should buy into a ideal - that further that it is objectively the only ideal, there being only one reality - that is ultimately for the benefit of those at the top of the pyramid, that represent the ideal man - her interpretation of the free spirits - and that society should be constructed only for the benefit of these people. That all the problems of society come about due to rejecting this and suppressing the elite - who would magically solve all the world’s problems if they were given free reign to pursue their own selfish goals. Really, it’s like the difference between devil worship and atheism, the latter rejects religion itself while the former gives up and decides to join the comic relief.

–edit, Wow if I known that I would have made best of Reddit I would have actually spent more time editting the post for clarity rather than giving up half way through and deciding to go to bed. It’s a jumble to be sure.

Don’t Use Hadoop - Your Data Isn’t That Big

A still-very-relevant 9-year old article. Pandas has gone from strength to strength since he wrote that.

In terms of expressing your computations, Hadoop is strictly inferior to SQL. There is no computation you can write in Hadoop which you cannot write more easily in either SQL, or with a simple Python script that scans your files.

SQL is a straightforward query language with minimal leakage of abstractions, commonly used by business analysts as well as programmers. Queries in SQL are generally pretty simple. They are also usually very fast - if your database is properly indexed, multi-second queries will be uncommon.

Hadoop does not have any conception of indexing. Hadoop has only full table scans. Hadoop is full of leaky abstractions - at my last job I spent more time fighting with java memory errors, file fragmentation and cluster contention than I spent actually worrying about the mostly straightforward analysis I wanted to perform.

If your data is not structured like a SQL table (e.g., plain text, json blobs, binary blobs), it’s generally speaking straightforward to write a small python or ruby script to process each row of your data. Store it in files, process each file, and move on. Under circumstances where SQL is a poor fit, Hadoop will be less annoying from a programming perspective. But it still provides no advantage over simply writing a Python script to read your data, process it, and dump it to disk.

In addition to being more difficult to code for, Hadoop will also nearly always be slower than the simpler alternatives. SQL queries can be made very fast by the judicious use of indexes - to compute a join, PostgreSQL will simply look at an index (if present) and look up the exact key that is needed. Hadoop requires a full table scan, followed by re-sorting the entire table. The sorting can be made faster by sharding across multiple machines, but on the other hand you are still required to stream data across multiple machines. In the case of processing binary blobs, Hadoop will require repeated trips to the namenode in order to find and process data. A simple python script will require repeated trips to the filesystem.

How Big is Too Big for JSON?

From over 10 years ago (I’m sorting through my old bookmarks). A single object looks like this:

{
    "ACCTOUNT_NUMBER":"1234567890",
    "CUSTOMER_NAME":"ACME Products and Services, Inc.",
    "ADDRESS":"123 Main Street",
    "CITY":"Albuquerque",
    "STATE":"NM",
    "ZIP":"87101-1234"
}

He tested the usability of a given browser while it loaded between 1 and 1,000,000 such records.

From this test, I am considering the sweet spot to be around 10,000 records at (1.55MB). The maximum number of usable records I would push to a browser would be around 25,000 records (3.87MB). Keep in mind there are numerous factors to keep in mind when determining how many records you should return to your JavaScript application. The purpose of this test was to help identify a general maximum number for conversations around large record sets with JSON.

Would love to see an updated version of the tests.

The Universal Estimation Table

Estimate Actual Time
Very Easy 1 Hour
Easy 2 Hours
Quite Easy 4 Hours
Looks Quite Easy 6 Hours
Average 8 Hours
Looks Average 12 Hours
No Clue 16 Hours
Seems Complex 24 Hours
Complex 30 Hours
Very Complex 40 Hours
Can Take Some Time 48 Hours
Fuck 60 Hours
Yeah Looks Pretty Easy 80 Hours

Combine: “No Clue. Can take some time, but yeah… looks pretty easy” = 16 + 48 + 80 = 144 hours.

Source Unknown. See also: “Midwest Distances”

Fearless Leader of Elite Strike-Force Team Suspended from Practicing Law in Home State where he was Once Top Federal Prosecutor

This led him to miss a court hearing where his own lawyer sought to dismiss a billion-dollar defamation case against him. This is because the Strike-Force Honcho made “false and misleading” claims about the equipment used in the 2020 election on behalf of his Stable Genius client.

I wonder how much alcoholism this saga of shit has induced in folks who write political satire for a living. It just never ends.

The Schmidt Pain Index

This is the Schmidt Sting Pain Index, an eponymous and subjective measurement of the pain caused by bees, wasps, and ants (and other things in the order hymenoptera.) It ranges from 0-4. In Level Zero, you don’t feel any pain whatsoever; the stinger doesn’t even penetrate your skin. The humble and familiar honeybee will deliver a Level 2.

Schmidt describes Level 4, the absolute worst, as follows:

Bullet Ant

“Pure, intense, brilliant pain… like walking over flaming charcoal with a three-inch nail embedded in your heel”

“That really shuts you down. It really felt like a bullet. It was instantaneous, almost even before it stung me. It was absolutely riveting. There were huge waves and crescendos of burning pain—a tsunami of pain coming out of my finger. The tsunami would crash as they do on the beach, then recede a little bit, then crash again. It wasn’t just two or three of these waves. It continued for around 12 hours. Crash. Recede. Crash. It was absolutely excruciating. There wasn’t much I could do except be aware of it and groan.”

Tarantula Hawk

“Blinding, fierce [and] shockingly electric”

“A running hair dryer has just been dropped into your bubble bath”

“Like you were walking underneath a high-voltage electric line in a wind storm, a wind gust snapped the line, and it fell on your arm. You get 20,000 volts all at once cascading through your body. It’s pure electrifying pain. Instantaneous. Very clean and sharp.”

Warrior Wasp

“Torture. You are chained in the flow of an active volcano. Why did I start this list?”

Here’s Dr. Schmidt with a giant bug on his face.

Dr Justin O. Schmidt

Some quotes and that image are from this article he penned in Esquire (cached) where he touches upon why the pain profiles are different.

“Crowded, Compartmentalized, Sticky, Spatially Inhomogeneous”

In college, I remember being blown away by a huge, physical map of metabolic pathways our Biochemistry professor once brought into class. It looked like this:

metabolic pathways

Here it is online. Kinda like a Google Maps of cellular reactions. It was impressed upon us that the interior of a cell (especially a eukaryotic one) is a really, really busy and tight and ‘goopy’ place: “crowded, compartmentalized, sticky, spatially inhomogeneous”. As that paper notes, this messy, “macromolecular crowding” helps your proteins fold properly (among several other factors.) This was a bit hard for me to appreciate since, up to then, I was only accustomed to images of cells from a light microscope or vastly simplified illustrations school texts.

I was somehow reminded of all of that after seeing some astounding paintings by Professor David S. Goodsell (Wiki, Twitter, Website). He calls the series “Molecular Landscapes.” Here are a few related to the pandemic we’ve been through.

SARS-CoV-2 and Neutralizing Antibodies, 2020

Art by David S. Goodsell

Acknowledgement: David S. Goodsell, RCSB Protein Data Bank and Springer Nature; doi: 10.2210/rcsb_pdb/goodsell-gallery-025. The painting was commissioned for the cover of a special COVID-19 issue of Nature, presented 20 August 2020, and is currently in the collection of the Cultural Programs of the National Academy of Sciences.

(Unknown Title)

Art by David S. Goodsell

Acknowledgement: Illustration by David S. Goodsell

Coronavirus, 2020

Art by David S. Goodsell

Acknowledgement: Illustration by David S. Goodsell, RCSB Protein Data Bank; doi: 10.2210/rcsb_pdb/goodsell-gallery-019. This painting depicts a coronavirus just entering the lungs, surrounded by mucus secreted by respiratory cells, secreted antibodies, and several small immune systems proteins. The virus is enclosed by a membrane that includes the S (spike) protein, which will mediate attachment and entry into cells, M (membrane) protein, which is involved in organization of the nucleoprotein inside, and E (envelope) protein, which is a membrane channel involved in budding of the virus and may be incorporated into the virion during that process. The nucleoprotein inside includes many copies of the N (nucleocapsid) protein bound to the genomic RNA.

SARS-CoV-2 Fusion, 2020

Art by David S. Goodsell

Acknowledgement: Illustration by David S. Goodsell, RCSB Protein Data Bank; doi: 10.2210/rcsb_pdb/goodsell-gallery-026. This painting depicts the fusion of SARS-CoV-2 (magenta) with an endosomal membrane (green), releasing the viral RNA genome into the cell cytoplasm (blue), where it is beginning to be translated by cellular ribosomes to create viral polyproteins. The painting includes speculative elements that are designed to highlight the process, most notably, multiple states of the viral spike protein are shown.

SARS-CoV-2 mRNA Vaccine, 2020

Art by David S. Goodsell

Acknowledgement: Illustration by David S. Goodsell, RCSB Protein Data Bank; doi: 10.2210/rcsb_pdb/goodsell-gallery-027. Messenger RNA (mRNA) vaccines developed for the COVID-19 pandemic are composed of long strands of RNA (magenta) that encode the SARS-CoV-2 spike surface glycoprotein enclosed in lipids (blue) that deliver the RNA into cells. Several different types of lipids are used, including familar lipids, cholesterol, ionizable lipids that interact with RNA, and lipids connected to polyethylene glycol chains (green) that help shield the vaccine from the immune system, lengthening its lifetime following administration. In this idealized illustration, all of the lipids are arranged in a simple circular bilayer that surrounds the mRNA and the PEG strands have both extended and folded conformations. In reality, the structure may be less regular, as suggested in the NanoLetters paper […]

“Windows is more than just an operating system; it’s a fabric woven into our lives and in our work.”

No. It’s just a fucking operating system1. A giant program on a computer. What weird, disconnected, embarrassing bullshit ‘spiritual’ techbro nonsense.

“We made a website full of memes you can search through, powered by Bitcoin and Machine-Learning (of course), and are changing the world.” No. Stop it.

I think I’m cranky because I’m hungry 🍕

  1. That cannot decide which decade it’s in. (Source)↩︎

Some Horoscopes

by Unknown

Aquarius (Jan 23 - Feb 22)
You have an inventive mind and are inclined to be progressive. You lie a great deal. You make the same mistakes repeatedly because you are stupid. Everyone thinks you are a fucking jerk.

Pisces (Feb 23 - Mar 22)
You are a pioneer type and think most people are dickheads. You are quick to reprimand, impatient and full of advice. You do nothing but piss-off everyone you come in contact with. You are a prick.

Aries (Mar 23 - April 22)
You have a wild imagination and often think you are being followed by the FBI or CIA. You have minor influence on your friends and people resent you for flaunting your power. You lack confidence and are a general dipshit.

Taurus (April 23 - May 22)
You are practical and persistent. You have a dogged determination and work like hell. Most people think your are stubborn and bullheaded. You are nothing but a goddamed communist.

Gemini (May 23 - June 22)
You are a quick and intelligent thinker. People like you because you are bisexual. You are inclined to expect too much for too little. This means your are a cheap bastard. Geminis are notorious for thriving on incest.

Cancer (June 23 - July 22)
You are sympathetic and understanding to other people’s problems, which makes you a sucker. You are always putting things off. That is why you will always be on welfare and won’t be worth a shit. Everyone in prison is a Cancer.

Leo (July 23 - Aug 22)
You consider yourself a born leader. Others think you are an idiot. Most Leos are bullies. You are vain and cannot tolerate criticism. Your arrogance is disgusting. Leo people are thieving motherfuckers and enjoy masturbation more than sex.

Virgo (Aug 23 - Sept 22)
You are the logical type and hate disorder. Your shit-picking attitude is sickening to your friends and co-workers. You are cold and unemotional and often fall asleep while fucking. Virgos make good bus drivers and pimps.

Libra (Sept 23 - Oct 22)
You are the artistic type and have a difficult time dealing with reality. If you are a male you are probably queer. Chances for employment and monetary gain are nill. Most Libra women are whores. All Libras die of venereal disease.

Scorpio (Oct 23 - Nov 22)
You are the worst of the lot. You are shrewd in business and cannot be trusted. You shall achieve the pinnacle of success because of your total lack of ethics. You are the perfect son-of-a-bitch. Most Scorpios are murdered.

Sagittarius (Nov 23 - Dec 22)
You are optimistic and enthusiastic. You have a reckless tendency to rely on your luck since you have no talent. The majority of Sagittarians are drunks. You are a worthless piece of shit.

Capricorn (Dec 23 - Jan 22)
You are conservative and afraid of taking risks. You are basically chickenshit. There has never been a Capricorn of any importance. You should kill yourself.

Friendly’s Staff Fails To Live Up To Restaurant Name, Writes “100% Sh*t Show” On Receipt

by Chris Morran

I read this story a long while ago and have been searching for it since. I have related increasingly exaggerated variants of it from memory every few years to friends who laugh nervously when I lose my damn mind to the “She just went home I guess” part.

To me, few things are funnier than stories of the wittingly incompetent and their inadvertent courage in this boring, rigged dystopia we live in.

A New Hampshire family says that after a night out to eat at Friendly’s didn’t go so well, a restaurant staffer decided to express their inner feelings through the increasingly preferred medium of the restaurant’s billing system, dropping the phrase “100% sh*t show” at the bottom of the dinner bill.

However, judging by the family’s recollection of the night, that phrase might be an accurate description of the service they received.

After waiting for about 30 minutes without service, they thought they finally located a waitress to take their order. But over the 45 minutes that followed, they were twice served other diners’ food, while theirs was missing in action.

“Come to find out, the waitress that had taken our order never submitted it she just left,” one of the customers tells WHDH-TV, “She went home I guess.”

The restaurant apologized and offered to pay for the family’s meal. And while this comp was reflected on the final check, so was the “sh*t show” remark.

“They had to have had it either entered into the cash register or they had a [expletive] show button,” says the diner, adding that she and her family aren’t offended or boycotting the restaurant and that they will go back to that Friendly’s in the future.

For what it’s worth, Friendly’s HQ released a statement to WHDH:

“[T]his type of behavior is completely unacceptable. We are investigating this with the restaurant and will take swift action. We hope we get the opportunity to rectify this directly with the guest.”

Thanks to Craig for the tip!

Two Messages for Father’s Day

Happy Father’s Day to the fathers, stepfathers, grandfathers, and father figures who enrich our character, love us unconditionally, and give so much of themselves every day so we can live lives worthy of their dreams and sacrifices.

President Joseph R. Biden Jr.

And, this is real (as much as you hope it isn’t but know deep-down that it is):

Happy Father’s Day to all, including the Radical Left, RINOs, and other Losers of the world. Hopefully, eventually, everyone will come together!

A soon to be ‘reinstated’, former President

I do love the capitalized “Losers”.

You Cannot Overcook Mushrooms

Note that you can certainly burn them. That’s not ‘cooking’, however. The key here is that mushroom cell walls are composed of chitin which is far more heat-stable by virtue of the structures it forms, compared to pectin which is what you’d find in veggies1.

In this video, Dan Souza explains all this and does something quite surprising when cooking mushrooms: He sautés the mushrooms in water to ‘collapse’ them prior to cooking them in just a teeny bit of oil (and the usual salt, pepper, butter, and herbs.) Amazing.

  1. You’ll also find chitin in the “exoskeletons of arthropods, such as crustaceans and insects, the radulae of molluscs, cephalopod beaks, and the scales of fish and skin of lissamphibians.” Leave it to fungi to be weird 😍🍄↩︎

Are they not Mothers and Fathers and Children?

I finish just by saying this: war is an easy thing to talk about; there are not many people - a - of the generation that remember it. The right hon. Member for Old Bexley and Sidcup served with distinction in the last war. I never killed anyone but I wore uniform. But I was in London in the blitz in 1940, living in the Millbank tower, where I was born. Some different ideas have come in since. And every night, I went down to the shelter in Thames house. Every morning, I saw dockland burning. Five hundred people were killed in Westminster one night by a land mine. It was terrifying. Aren’t Arabs terrified? Aren’t Iraqis terrified? Don’t Arab and Iraqi women weep when their children die? Does bombing strengthen their determination? What fools we are to live in a generation for which war is a computer game for our children and just an interesting little channel for news item.

Every Member of Parliament tonight who votes for the Government motion will be consciously and deliberately accepting responsibility for the deaths of innocent people if the war begins, as I fear it will. Now that’s for their decision to take. But this is a quite unique debate. In my parliamentary experience, where we are asked to share responsibility for a decision we won’t really be taking, with consequences for people who have no part to play in the brutality of the regime which we are dealing with.

And I finish with this: on 24 October 1945—the right hon. Member for Old Bexley and Sidcup will remember—the United Nations charter was passed. And the words of that charter are etched into my mind and move me even as I think of them. “We the peoples of the United Nations determined to save succeeding generations from the scourge of war, which twice in our life-time has caused untold suffering to mankind”. That was the pledge of that generation to this generation, and it would be the greatest betrayal of all if we voted to abandon the charter, and take unilateral action and pretend that we were doing it in the name of the international community. And I shall vote against the motion for the reasons that I have given the house.

Tony Benn, on 17 February 1998, Westminster.

Using AWS Without Succumbing to Hype, FOMO, and Over-Engineering

by Daniel Vassallo

This is how I use the good parts of @awscloud, while filtering out all the distracting hype.

My background: I’ve been using AWS for 11 years — since before there was a console. I also worked inside AWS for 8 years (Nov 2010 - Feb 2019).

My experience is in web- sites/apps/services. From tiny personal projects to commercial apps running on 8,000 servers. If what you do is AI, ML, ETL, HPC, DBs, blockchain, or anything significantly different from web apps, what I’m writing here might not be relevant.

Step 1: Forget that all these things exist: Microservices, Lambda, API Gateway, Containers, Kubernetes, Docker.

Anything whose main value proposition is about “ability to scale” will likely trade off your “ability to be agile & survive”. That’s rarely a good trade off.

Start with a t3.nano EC2 instance, and do all your testing & staging on it. It only costs $3.80/mo.

Then before you launch, use something bigger for prod, maybe an m5.large (2 vCPU & 8 GB mem). It’s $70/mo and can easily serve 1 million page views per day.

1 million views is a lot. For example, getting on the front page of @newsycombinator will get you ~15-20K views. That’s just 2% of the capacity of an m5.large.

It might be tempting to use Lambda & API Gateway to save $70/mo, but then you’re going to have to write your software to fit a new immature abstraction and deal with all sorts of limits and constraints.

Basic stuff such as using a cache, debugging, or collecting telemetry/analytics data becomes significantly harder when you don’t have access to the server. But probably the biggest disadvantage is that it makes local development much harder.

And that’s the last thing you need. I can’t emphasize enough how important it is that you can easily start your entire application on your laptop, with one click.

With Lambda & API Gateway you’re going to be constantly battling your dev environment. Not worth it, IMO.

CloudFormation: Use it. But too much of it can also be a problem. First of all, there are some things that CFN can’t do. But more importantly, some things are best left out of CFN because it can do more harm than good.

The rule of 👍: If something is likely to be static, it’s a good candidate for CFN. Ex: VPCs, load balancers, build & deploy pipelines, IAM roles, etc. If something is likely to be modified over time, then using CFN will likely be a big headache. Ex: Autoscaling settings.

I like having a separate shell script to create things that CFN shouldn’t know about.

And for things that are hard/impossible to script, I just do them manually. Ex: Route 53 zones, ACM cert creation/validation, CloudTrail config, domain registration.

The test for whether your infra-as-code setup is good enough is whether you feel confident that you can tear down your stack & bring it up again in a few minutes without any mistakes. Spending an unbounded amount of time in pursuit of scripting everything is dumb.

Load balancers: You should probably use one even if you only have 1 instance. For $16/mo you get automatic TLS cert management, and that alone makes it worth it IMO. You just set it up once & forget about it. An ALB is probably what you’ll need, but NLB is good too.

Autoscaling: You won’t need it to spin instances up & down based on utilization. Unless your profit margins are as thin as Amazon’s, what you need instead is abundant capacity headroom. Permanently. Then you can sleep well at night — unlike Amazon’s oncall engineers 🤣

But Autoscaling is still useful. Think of it as a tool to help you spin up or replace instances according to a template. If you have a bad host, you can just terminate it and AS will replace it with an identical one (hopefully healthy) in a couple of minutes.

VPCs, Subnets, & Security Groups: These may look daunting, but they’re not that hard to grasp. You have no option but to use them, so it’s worth spending a day or two learning all there is about them. Learn through the console, but at the end set them up with CFN.

Route 53: Use it. It integrates nicely with the load balancers, and it does everything you need from a DNS service. I create hosted zones manually, but I set up A records via cfn. I also use Route 53 for .com domain registration.

CodeBuild/Deploy/Pipeline: This suite has a lot of rough edges and setup can be frustrating. But once you do set it up, the final result is simple and with few moving parts.

Don’t bother with CodeCommit though. Stick with GitHub.

Sample pipeline: A template for setting up an AWS environment from scratch.

S3: At 2.3 cents per GB/mo, don’t bother looking elsewhere for file storage. You can expect downloads of 90 MB/s per object and about a 50 ms first-byte latency. Use the default standard storage class unless you really know what you’re doing.

Database: Today, DynamoDB is an option you should consider. If you can live without “joins”, DDB is probably your best option for a database. With per-request pricing it’s both cheap and a truly zero burden solution. Remember to turn on point-in-time backups.

But if you want the query flexibility of SQL, I’d stick with RDS. Aurora is fascinating tech, and I’m really optimistic about it’s future, but it hasn’t passed the test of time yet. You’ll end up facing a ton of poorly documented issues with little community support.

CloudFront: I’d usually start without CloudFront. It’s one less thing to configure and worry about. But it’s something worth considering eventually, even just for the DDoS protection, if not for performance.

SQS: You likely won’t need it, and if you needed a message queue I’d consider something in-process first. But if you do have a good use case for it, SQS is solid, reliable, and reasonably straightforward to use.

Conclusion: I like to seperate interesting new tech from tech that has survived the test of time. EC2, S3, RDS, DDB, ELB, EBS, SQS definitely have. If you’re considering alternatives, there should be a strong compelling reason for losing all the benefits accrued over time.

On Software Engineering and Complexity

me, a software engineer: large scale production systems are complex and require teams of experts to keep running. It’s near impossible to get right 100% of the time

me, when an app I use goes down once: these fucking clowns, what the fuck

@aweary

Floofballs

Vox on something I’ve always wondered: Why tennis pros reject balls before a serve (and what happens to the ones they do.)

More fluff ⇒ more drag ⇒ more time for your opponent to react. So they’re looking for a ball with less fluff for their first serve, and a floofier one for their second.

But while it is provably true that floofier balls are slower, and as Serena Williams’ coach points out, the more important (and potent) thing here is the mental effect of the ritual itself and not the actual Physics 🎾

The Medieval Friendzone

A young Elizabeth I found herself on the throne of England immediately “besieged by suitors” to whom she made “no firm promises” but sent very nice-sounding letters. One such suitor was a young Eric XIV of Sweden. He was so thirsty, he offered to come to England to visit her. That’s when she fired off this missive on the 25th of February, 1559.

Elizabeth I to King Eric XIV of Sweden

Translation:

Most Serene Prince, our very dear Cousin,

A letter truly yours both in the writing and sentiment, was given us on 30 December by your very dear brother, the Duke of Finland. And while we perceive therefrom that the zeal and love of your mind towards us is not diminished, yet in part we are grieved that we cannot gratify your Serene Highness with the same kind of affection. And that indeed does not happen because we doubt in any way of your love and honour, but, as often we have testified both in words and in writing, that we have never yet conceived a feeling of that kind of affection towards any one. We therefore beg your Serene Highness again and again that you be pleased to set a limit to your love, that it advance not beyond the laws of friendship for the present nor disregard them in future… I have always given both to your brother, who is certainly a most excellent Prince and deservedly very dear to us, and also to your ambassador likewise, the same answer with scarcely any variation of the words, that we do not conceive in our heart to take a husband but highly commend the single life, and hope that your Serene Highness will not longer spend time in waiting for us.

Source
Mare of Easttown

Mare of Easttown (2021)

IMDb

Rating: A+

Watched with LD. Kate Winslet is an amazing actor and this miniseries is her best work yet. It is excellent everything: story, score, cinematography, dialogue, casting, acting. About economic depression, blue-collar America, forgotten America, the opioid epidemic caused by unchecked Capitalist greed, community, family, single parenthood, decline in religious participation, and the country’s abysmal attitude towards the treatment of mental illness.

That’s a lot of layers and facets and it’s all done exquisitely, gut-wrenchingly well.