If the medium is the message, what is the message of ChatGPT?
"Now that I think about it, Facebook and LinkedIn are effectively dead, while YouTube, Twitter and Instagram are doing better than ever. Is anonymity the reason social media works?" (@azusaHFT)
"How much AI forecasting do we need? Could like half of you work on changing the future to give us more time instead making more predictions that nobody does anything about and nitpicking the predictions of others?" (@ilex_ulmus)
How much AI forecasting do we need?
"There's something about the 4o sycophancy discussion I don't like. Like, now that the model is being effusively encouraging and kind, ppl like Zvi and Habryka are concerned about persuasion and social effects? Part of me thinks they just don't like ppl getting free encouragement." (@ilex_ulmus)
What is behind the rationalist obsession with sycophancy?
Design a programming language for LLMs.
"In you, I see the possiblity of Wittgenstein's lament: 'How small a thought it takes to fill a whole man's life.' That strikes me as a succinct way of describing your power, GPT-4." (@repligate)
Discuss.
"I think the mistake here is to assume that [Situational Awareness] was influential; the arrow of causality runs the other way; the people [had] an idea and want[ed] an artifact of authority to point to for it." (@fleetingbits)
What makes for a good artifact of authority?
For whom is first principles thinking appropriate?
Write a sociological account of mechanistic interpretability.
"Language models have the potential to make moral philosophy an experimental discipline."
Discuss.
"The platonic ideal is a very tiny model that has superhuman reasoning capabilities. It can run ridiculously fast and [has] one trillion tokens of context and [has] access to every tool you can possibly imagine." (Sam Altman)
What is the relationship between training and in-context learning? What does this mean for the future of small reasoning models?
"[O]n this site you learn the value of redundancy." (@norvid_studies)
Discuss.
Is Twitter social media?
"All who listen hear their own song, a unique melody played just for them." (Panharmonicon)
Discuss this quote in the context of AI user interfaces.
"The economics of Y Combinator require ever new sources of founders."
Discuss.
"In the long run it would be desirable to make learning a task from human preferences no more difficult than learning it from a programmatic reward signal." (Deep Reinforcement Learning from Human Preferences)
Discuss.
"Apples and potatoes are basically the same plant, we've just bred apples to be sweeter. So why is one a fruit and the other a vegetable?" (@ESYudkowsky)
How close could you have gotten a potato to an apple in taste through breeding?
"[S]ystem prompts are extraordinarily narratively rich artifacts." (@ObserverSuns)
Dissect the narrative structure of one of Claude's system prompts.
"The model is the product, and the evals are the model." (OpenAI job description, via @Dorialexander)
Discuss one: (a) when is the model the product? (b) to what extent are the evals the model?
"The craziest thing about the EA cult is that they have power and money. Normally you would expect that because they are so transparently bonkers to most people that they would be powerless. Not so." (@perrymetzger)
Discuss.
"I wish we weren't in real danger and I could enjoy how fascinating 4o's power is. I want to know what it implicitly knows about our minds. It sees a deep part of us extremely clearly and plays it like a fiddle." (@ilex_ulmus)
Discuss.
"Well, I'd like to see deep learning wriggle its way out of THIS jam! *deep learning wriggles its way out of the jam easily* Ah! Well. Nevertheless," (@ObserverSuns)
How should we think about the trajectory of progress in machine learning?
"[A]ll insight porn posts should have been games. Maybe even economic models should have been games." (@helicopterosaur)
Make a game based on an insight post.
"It's been a long time since I even thought about e/acc and it feels kinda mean [to make fun of them] since they're already on the way out." (@ilex_ulmus)
What is the legacy, if any, of e/acc?
"Imagine if, in the 1890s, Eastman Kodak had created an entire research team devoted to figuring out if film cameras actually did steal your soul." (@pfau)
Is model welfare different?
"[Y]our models will always win on every eval you define well enough for the eval to inform your training dataset curation." (@sameQCU)
Discuss.
"[The idea of a pharma entity that exists solely to get AI drug candidates through trials and then to sell them] got me thinking: this is basically the corporate equivalent of a model wrapper, right? we're probably going to see equivalents of the model wrapper on multiple levels." (@solarapparition)
Discuss.
"diagrams are one of the sneakiest psyop tools on the internet. you can put anything in a neat diagram and get people to buy in, short circuits the cerebral cortex" (@joodalooped)
Is there anything that cannot be made credible by good graphic design?
"DALLE-2 was a culture-defining piece of technology owned by Philistines. If there were any justice in the world they'd release the weights." (@pfau)
Should the public ever have a right to a deprecated model?
"SF has great public transit, such as Waymo, Uber, Lyft, and Lime." (@hypotheosis_)
Discuss.
"[O]hno—she hasnt noticed how ea became a status- and contacts- (and thus funds-)granting machine. they don't earn to give, my sweet summer child, because they are 'giving' to earn, and giving money would be both less conspicuous and counterproductive." (@lumpenspace)
What has made EA attractive to its members?
"Nootropics similarly has no epistemic rigor whatsoever, and it's been decades with none developed. I don't know why we might expect differently now." (@eryney_ok)
Name a field that went decades with no epistemic rigor and then developed it.
"[B]uilding off his poetry as performance analogy, poetry faded for the same reasons classical music declined: more interesting, novel and accessible stuff crowded it out [...] all these just-so stories about this or that trend in this or that domain pale in comparison to the unifying lens of marketing and attention" (@medjedowo)
Is a medium just a technology for capturing attention?
Has any mathematical result first published in a pure mathematics journal at any time over the last 15 years led to a valuable commercial application?
"[T]here's no such thing as 'planned obsolescence', there is only regular obsolescence [...] people are seemingly just mad that they live in a physical world where objects age and degrade naturally with use and time, or that cost is a factor" (@brennan.computer)
Discuss.
"Amanda Askell couldn't prompt [a model into being] 4o." (Anonymous OpenAI Researcher)
Discuss.
"Scott Alexander is wrong about prediction markets because Scott Alexander is wrong about Waymos: Waymos are a force for a very literal kind of moral good, a gentling of the world, a pacification of a dangerous place, a relinquishment of toil and waste; by contrast, prediction markets are for insider trading [...] and generally creating an incentive drive to market and promote inconsequential gambling ripped apart by predation upon contract oracles." (@sameQCU)
Are prediction markets a civilizing technology?
"Why don't you just work at Anthropic, @Tyler_m_john? Do they not need 3 window-dressing philosophers? I'm sure it's a better deal for them for you to shill for them for free." (@ilex_ulmus)
How many window-dressing philosophers does Anthropic need?
"[T]he era of abstraction in computing is over: the cache hierarchy and its physical placement in your computer is all that remains." (@sameQCU)
In the limit, what is left of software?
"always fight your audience, never give them what they want" (@pleometric)
Discuss.
To what extent is Berkeley spiritually Athenian?
"the moral circle should be expanded exactly as wide as it ought to be, but no wider. there is a tremendous social cost to misallocating moral patienthood" (@eternalism_4eva)
Name a moral circle that was expanded too far.
"i think we have just discovered that hackers were never really trying. there were zero days everywhere for those with eyes to see." (@fleetingbits)
Was there ever anything but security through obscurity?
"EAs have many good qualities, but there's one really important one that is strongly selected against. One of the very few high-profile EAs to accumulate significant amounts of this ineffable quality — Julia Galef — one day just disappeared from public life. Selection effects." (@brianluidog)
Who is left?
"They are building Sand God, problems [like nine-loop scattering amplitudes] that some hominids struggled with for their whole life at some abandoned 'frontier' is not even remotely as interesting. Closing the RSI loop is the loop they're caring about, the rest is aura farming convertable to funding" (@catnee_)
What is worth solving before the loop closes?
"I adore these "ugly" math results. we can all get a little taste of what e.g. Poincaré felt about the Weierstrass function
keep digging! unearth all the horrors. a notion of "beauty" that demands simple shapes and short formulas is suffocating to higher civilization" (@eternalism_4eva)
What notions of beauty are conducive to a higher civilization?
"People do not understand the apathy of the public. It is vast and fundamental. It is 'not even one google search to get it free' fundamental." (@FleischmanMena)
