Gleanings from Double Crux on “The Craft is Not The Community”

Epistemic status: This is a bunch of semi-remembered rephrasings of a conversation.

At the CFAR alumni reunion, John Salvatier and I had a public double crux on my last post.

A double crux is a technique CFAR invented, which I think is much better than a debate. The goal is to simply pin down where exactly two people disagree. This can take a while. Even the best, most respectful debates are adversarial: it’s my opinion vs. yours, and we see which is stronger in an (ideally fair) contest. A double crux is collaborative: we’re just trying to find which is the exact point of contention here, so that if we go on to have an actual debate we won’t be talking past each other.

John’s motivation for disagreeing with my post was that he didn’t think I should be devaluing the intellectual side of the “rationality community”. My post divided projects into into community-building (mostly things like socializing and mutual aid) versus outward-facing (business, research, activism, etc.); John thought I was neglecting the importance of a community of people who support and take an interest in intellectual inquiry.

I agreed with him on that point — intellectual activity is important to me — but doubted that we had any intellectual community worth preserving.  I was skeptical that rationalist-led intellectual projects were making much progress, so I thought the reasonable thing to do was to start fresh.

John is actually working on an intellectual project of his own — he’s trying to explore what the building blocks of creative thinking are, and how one can improve it — and he thinks his work is productive/useful, so that seemed a good place to dig in deeper.

I mentioned that by a lot of metrics, his work doesn’t have a lot of output. He has done a lot of one-on-one conversations and informal experiments with people in the community, but there’s no writeup, and certainly no formal psychological research, papers, or collaboration with psychologists. How could an outsider possibly tell if there’s a real thing here?

John said that I might be over-valuing formality. He’s pretty confident that the “informal” phase of work — the part when you’re just playing with an idea, or planning out your strategy, before you sit down to execute — is actually the most important part, in the sense that it’s highest-leverage. After some discussion, I came to agree with him.

I’ve definitely had the experience that creative work is “bursty” — that most days you produce piles of junk, and some days you produce solid gold, whether it’s writing, math, or code. I’ve also heard this from other people, both friends and famous historical figures.  It also seems that when something’s going right about your “pre-work” cognitive processes — planning, imagining, even emotional attitudes — you do much better work at the formal, sit-down-and-produce-output stage.  Work goes hugely better when the “muse” is friendly.

John additionally believes that it’s possible to “train your muse” to help you work better, and said that learning to do this himself allowed him to contribute much better to open-source software projects (where he built a statistics library.)

He also pointed out that when it comes to dealing with the distant future, general-purpose and speculative cognitive processes will have to be more important than trained skills, because the future will contain unfamiliar situations that we haven’t trained for. People who excel at the sit-down-and-execute activities that help you succeed in your field aren’t necessarily going to be able to reason about the weirdness of a changing world.

(I agreed that the ability to “philosophize” well seems to be much rarer than the ability to execute well; I’ve seen many prominent computer scientists whose theories about general intelligence just don’t make sense.)

So the speculative, philosophical, imaginative stuff that comes before sitting down and executing is important for success, important for humanity, and maybe something we can learn to do better. John certainly thinks so, and wants the rationality community to be a sort of laboratory or nursery for these ideas.

It’s also true that formally executing on these ideas can be really hard, if you define “formally” strictly enough. Here’s Scott Alexander reflecting on the bureaucratic hell of trying to get a psychiatry study on human subjects approved by an IRB — when it only involved giving them a questionnaire!  If that’s what it takes to do academic experimental research on humans, I don’t want to claim that anybody who’s thinking about the human mind without publishing papers can be rounded down to “doing nothing.”

That still leaves us with the question of “how do I know — not an IRB, not the ‘general public’, but Sarah, your friendly acquaintance — that you’re making real progress?”  I’m still going to need to be shown some kind of results, if not peer-reviewed ones.  This is why I’m a fan of blogging and something in the neighborhood of “citizen science.”  If a programmer tests the speed of two different programs and writes up the results, code included, I believe them, and if I’m skeptical, I can try to duplicate their results. It’s in the spirit of the scientific method, even if it’s not part of the official edifice of Science(TM).

So, John and I still have an unresolved disagreement about the general status of these “how to think real good” projects in the community.  He thinks they’re moving forward; I still haven’t seen evidence that convinces me.  This is our “double crux” — both of us agree (the “double” part) that it’s the key (“crux”) to our disagreement.

But I definitely agree with John that if there were promising ways to “think real good” being developed in our community, then it would be important to support and encourage that exploration.

One interesting thing that we had in common was that we both viewed “community” from a strongly individualist standpoint. John said he would evaluate someone as a potential collaborator on a project pretty much the same way whether they were a community member or not — track records for success, recommendations from friends he respects, and so on.  The “community” is useful because it’s a social network that sometimes floats cool people to his attention.  Deeper notions of tribe or belonging didn’t seem to apply, at least concerning his intellectual aims.  He had no interest in kicking people out for not following community standards, or trying to get everybody in the community to be a certain way; if a person considered themselves “part of the community” but John couldn’t see benefit from associating with that person, he just wouldn’t associate.  This is not everybody’s point of view — in fact, some people might say that John’s idea of a community is equivalent to not having a community at all.  So a lot of the things that seem to spark a lot of debate these days — community standards, community norms, etc — just didn’t show up in this double-crux at all, because neither of us really had strong intuitions about governance or collective issues.

Mostly, I came away with a lot of food for thought about the reflection vs. execution thing.  If there’s a spectrum between musing about the thing and doing the thing, I’m pretty far towards the “musing” side relative to the general population, so I’d generally assumed that I do too much musing and not enough executing.  “Head-in-the-clouds dreamer” and “impractical intellectual” and all that.  (Introspection falls into this category too; thinking too much about your own psyche is “navel-gazing”.)  But reflecting well seems to be incredibly high-reward relative to the time and effort spent, for compounding reasons. Strategizing so that you work on the right project, or putting attention into your mental health now so that you’re systematically more productive in the future, has a much bigger impact than just spending one more marginal hour on the daily slog.  Reflecting and strategizing gave my friend Satvik much more success at work.

It’s always felt a little presumptuous to me — like “who am I to think about what I’m doing? I’m supposed to keep my head down, keep slogging, and not ask questions!  Isn’t it terribly selfish to wonder what helps me do my best, rather than just doing my duty?”  But that’s a set of norms that gets applied to children, soldiers, and laborers (and maybe it shouldn’t even then), not to people like me. My peers expect that a person who does “knowledge work” for a living and writes essays will, of course, reflect on what she’s doing.

So maybe I ought to be going back and reading what reflective people write, taking it seriously this time around. “The unexamined life is not worth living.”  What if you literally meant that?  What if thinking about stuff was not a half-forbidden luxury but the most important thing about being human?


7 thoughts on “Gleanings from Double Crux on “The Craft is Not The Community”

  1. I love the concept of the double crux, thank you for sharing it.

    I too have worried about doing too much reflection. What’s helped me has been coming up with stopping conditions, whether that’s coming up with a satisfying answer to a specific question, or a deadline, or both, or something else entirely. That way I’m not worried about accidentally losing my head in the clouds forever.

  2. I also wanted to make a comment about “double crux” more generally rather than the actual content of this post (sorry). So — when I first saw the “double crux” thing, my reaction was basically “Isn’t this basically how sensible people already argue?” But of course there’s value to naming the technique and making it known since not everyone is going to realize this is what they have to do, and of course naming it like this allows it to be isolated and done directly instead of being just the thing that’s essentially going on under the surface. (And like, being able to refer to it as a thing to do… I feel like a lot of arguments I’ve had previously have ended up as “single crux” so to speak and gone nowhere.)

    But anyway — public double crux! Now there’s something that really shows off the usefulness of the idea, and something I’d never considered before. Because a public event like that needs some sort of structure; you can’t really have a public “two people discuss things”. And if you have to have a structure, double crux is much better than the classic debate as you say. I hope that catches on more.

  3. The two clusters of things that seem relevant to me are:
    1) Is it possible to think-real-good, and is our community better at it than average? I do think the answer is yes, but not overwhelmingly so. (I think the community does encourage and help people level-up their epistemics, but it is not sufficiently good at it to bridge the gap between “okay thinker” and “great thinker” without a lot of separate effort, inclination and talent on the part of individuals)
    2) Are there things worth doing that *don’t* require exceptional quality of thought, but which simply require you to even be paying attention to the right things, and then being able to think *reasonably* well about them. This is what I think our community excels at. If we didn’t have extensive discourse around X-Risk and other weird topics, it would be a lot harder to get funding, personnel, etc to make credible progress on those things at all.
    I think the most important work (on X-Risk specifically but other challenging problems as well) requires both high-executive function and high-creative/rational-output, and this may not be accessible to the average community member. But a) having a centralized discussion of that is what allows people like Luke Muehlhauser to notice X-Risk is important, drop everything and get started working on it, and b) the field still benefits from a lot of pretty-smart people who the community helps be somewhat-smarter who are contributing to the overall discussion.
    (Possible cruxes: X-risk matters, and it’s possible to make credible progress on it)

  4. Side note: the editing of comments here got more weirdly formatted than the last time I participated (I think the formatting of comments-in-edit-mode and final-comments used to be the same, and now comments-in-edit-mode have weird line-height that makes inserting paragraph breaks look weird)

  5. Lots of great stuff here! I look forward to our in-person discussion this weekend.

    I think all of us are in agreement that it is the intellectual project (the second foundation, not the first), that it is the one that counts. The key double crux of ‘are our intellectual projects making progress?’ is a question I am uncertain of. I am confident that there exist projects on which we make progress, but other places we are losing progress, and I worry greatly that our ability to make progress – and our ability to sustain and grow the systems that make progress – are decaying.

    This isn’t a full crux for me, because while it is *necessary* to change my mind and it introduces uncertainty, it’s not *sufficient*. Even if we are failing, or even have failed, I’m not giving up on the mission. Tis a far, far better thing I do, death always wins but you have to fight him anyway (and maybe one day…), and so forth.

    If the answer is yes, we are becoming stronger, then that’s great. I would love that. Then the path forward is to further that project.

    If the answer is no, then the question becomes, is what we have good enough, and sufficiently free from contamination (of whatever sorts), to be worth fighting for? This isn’t an easy project, and I don’t see anyone else covering themselves in glory around these parts. Our failure to be making progress would not be sufficient evidence for me to say, this is a dead end, we gotta go.

    So once again, if the answer is this is the best shot we have, we fight for it, to save it. Then we talk about what’s causing us to fail.

    If the answer is, the project has failed, then the questions are: How do we start over, and what is worth preserving?

    Another way of thinking about this is, do we actually have a tradition that’s worth a damn? Or the components necessary to construct one? My answer to that is an emphatic YES. I have a tradition. If it exists only in my head, and/or only in pieces, then it is my sacred duty to assemble it and share it. The me here generalizes.

    Raymond’s hypothesis is interesting. The art of think-real-good is rare even for moderate levels of real-good, let alone think-insanely-great. It’s important to recognize how low the standards really are. One could model what we’re doing as a way to think-real-good from scratch, on a low budget (in time, money, status, you name it, all at the same time), and for giving people the chance to learn how to do it too (I think that the goal of ‘get random willing person to think-real-good without them doing a lot of work is stupidly hard and certainly unsolved). Getting people to think-great is always going to be even more rare, and that’s fine, but we want it to be rare and not mythical maybe-never-see-it rare.

    What I found even better about this post was that it has (at least) two great illustrations of why one would want to care a lot about the community and community standards.

    First, we have John’s finding people to collaborate with by using the community to bubble people up to the surface. Yes, you could say, I don’t care about person who isn’t good enough, I just won’t associate with them. But what this does is, it weakens the search space. John has to consider more people to find a good one; John’s prior shifts. The value to John of meeting new people, of going to parties, goes down. John still cares about the community, and his solution – don’t associate with anyone who isn’t good enough – would in fact be a sufficient general strategy for keeping high standards. It’s kind of the definition of high standards! So one could say, he has the norm others want him to have, he just doesn’t think it’s a norm, so he won’t enforce it.

    The other is Sarah’s note of her own perspective at the end. Anchoring bias is huge! Check this out:

    “It’s always felt a little presumptuous to me — like “who am I to think about what I’m doing? I’m supposed to keep my head down, keep slogging, and not ask questions! Isn’t it terribly selfish to wonder what helps me do my best, rather than just doing my duty?” But that’s a set of norms that gets applied to children, soldiers, and laborers (and maybe it shouldn’t even then), not to people like me. My peers expect that a person who does “knowledge work” for a living and writes essays will, of course, reflect on what she’s doing.

    So maybe I ought to be going back and reading what reflective people write, taking it seriously this time around. “The unexamined life is not worth living.” What if you literally meant that? What if thinking about stuff was not a half-forbidden luxury but the most important thing about being human?”

    There’s also a side point about ‘wondering what helps me do my best’ being the least selfish thing one can do, since it allows you to do your duty better, unless you think your duty is to either satisfice or to signal having done duty, rather than actually helping anyone. I don’t see doing this kind of duty as helpful.

    The key point, though, is that Sarah is (quite humanly and normally) anchoring on the norms and attitudes around her. She’s implicitly assuming that where she differs from the crowd, some mixture of her view and the crowd is probably right, or the best guess. So if Sarah hangs out with a bunch of people who don’t care about thinking as much (which given who is using the term ‘thinking’, is EXACTLY the thing I am worried about) or even sees them around town, she will conclude she has no right to value thinking so much. If they care a lot about some issue in social justice, or politics, or California football, she will feel she needs to care more about social justice, politics and California football. Ra. Ra. Ra.

    This is exactly why the norm has shifted from “keep your donations secret” to “make your donations public.” Jewish tradition, like many others, says the best gift is so secret that not even the person getting the money knows it was you, because this avoids status games and people warping their behavior to satisfy donors, and forces people to do good rather than show good. EA tradition says, that’s all well and good, but it’s dwarfed by people seeing you do and give, and adjusting their norms towards doing and giving more. I think both sides here make good points.

    What we need is for Sarah to hang out and around people who believe we should value (on every meta level) such thinking MORE, not less. Ideally, they should aspire to care about it more than they do in fact care, so as to slowly raise themselves up in this way! Then we will look at this standard and think, maybe it’s our duty to think better/more, not nicer/less. This will only happen if you properly engineer your surroundings, and the most important surrounding is other people and the norms/culture they follow.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s