On Reading Papers

My motive for reading any given paper is generally not “for the sake of reading the paper from start to finish”. And these days I am not reviewing any papers for journals or conferences. So I start by identifying my purpose.

Prerequisite #0: Define your purpose.

  • Am I trying to learn a new area of expertise?
  • Am I working on a design and want to understand other approaches?
    • Am I trying to identify best practices?
    • Am I trying to identify and evaluate the correctness, efficiency, and usability of the newest research in a given niche?
  • Am I working on a specific problem and want to understand how others have solved related problems?
  • Do I have a specific list of questions that I am trying to answer? Should I draft such a list?

Prerequisite #1: Find a paper to consider reading.

  • If you are trying to learn a new area:
    • Try to find a dissertation on the topic, preferably recent, but older dissertations can be quite useful launch points, both for understanding and for further reading.
      • Dissertations usually contain background chapters that present the material in an intelligible way to a non-expert (where here, non-expert just means “someone not in the niche Y of discipline X”).
      • Even if the related work section is dated, dissertations often include pointers to the seminal papers in the field, which can be extremely helpful to read.
    • Find the seminal paper or papers in the field. Use a search engine that can give you a list of work that cites these papers. Start with more recent citations, surveys, and systematizations of knowledge (SoKs).
    • Identify top venues.
  • If you are not (completely) new to the area:
    • Track publications at top venues. If your job involves staying on top of current research, devote some time to a cursory read of titles and abstracts. Also see Guideline #3.
    • Devote some time, regularly, to a cursory perusal of pre-print abstracts.
    • Join a forum that has an active reading discussion group.

Guideline #1:First make sure you know whether and why you want to read the given paper.

  • Why do I think I should consider reading the specific work in question, e.g.,
    • Am I trying to learn a new area of expertise?
      • Does the paper contain a comprehensive overview of the literature, or at least friendly-looking background and related works sections? (Papers that do not have these features are, in my experience, not the best place to start.)
      • Does the list of references seem comprehensive and recent? Do I think the paper will act as a guide for further reading?
      • Is the paper a seminal work in the field? Do lots of other papers rely on it?
    • Where was the paper published? Who are the authors? Do I have any faith in the quality level of the paper based on this information?
    • Am I deciding to implement a protocol related to the one developed and analyzed in the paper?
      • The protocol from the paper itself?
      • Or a protocol that solves the same or a similar problem?
    • Am I trying to solve a related problem?
    • Would it suffice to understand the claims in the paper at a high level, contextualized against other, related papers?
  • Why do I want to read the specific work in question, e.g.,
    • Does the paper seem interesting or well-written?
      • Does it contain implementation considerations and pitfalls?
      • Does it claim to solve a problem I think is cool or particularly impactful?
    • Are the authors likely to have a unique perspective on the topic?
    • Do I know the authors personally and want to support them by engaging with their work?
    • Do I know the authors from previous work and know from experience that I am likely to glean something useful from reading the paper?

Guideline #1 and Prerequisite #0 have some overlap, but I strongly believe it is useful to iterate on your understanding of your purpose through repetition.

Guideline #2: Reading a paper thoroughly from start to finish is often not the most effective way of accomplishing your purpose.

S. Keshav has a page on reading papers in three passes. This is a good place to start. I tend to read papers in multiple passes, sometimes over the course of years. Here is my routine:

  • Pass #0: Many papers don’t get more than a casual perusal of the abstract, authorship list, and publication venue. Papers most often remain in this category:
    • When the paper sounds just vaguely interesting because of the title, venue, or authorship list, but I don’t have any real need, interest, or time to delve deeper.
    • When the answer to the question “Would it suffice to understand the claims in the paper at a high level, contextualized against other, related papers?” is Yes. Hopefully the abstract spells out the claims in sufficient detail; I can always return to this paper later: see Guideline #3.
  • Pass #1:
    • Familiarize yourself with the structure of the paper: what are the headers and subheaders? Can you already tell what the paper claims to show? If so, identify the claims explicitly.
    • Read some bits and pieces: I start with the introduction, skim through any theorem statements or claims, and read the conclusion.
    • Identify the claims and research performed. At this point, I usually stop reading if I can’t identify the claims or experiments performed, or find the referenced implementation.
    • Evaluate plausibility of claims: Determining correctness can be difficult. I find it often involves a thorough reading of the paper and a relatively deep understanding of the literature and field in question. For this pass, I focus on plausibility.
      • Do I think the results are reasonable or persuasive?
      • Are there any red flags? e.g., Are graphs and figures intelligible? Are clear definitions and theorem statements given? Does the list of references seem reasonably complete? Does the full version of the paper actually exist? How about the implementation?
      • If I don’t know the field well, sometimes I leave this at “I can’t really tell, but at least it was peer-reviewed at a top venue in Field X”, etc.
  • Pass #2: Sometimes I check to see if a talk recording is available and watch it instead of, or as part of, this pass. And no, I don’t feel the need to watch the video from beginning to end or at normal speed. But on this pass:
    • Contextualize the paper against related work: Are the claims as interesting as the abstract and introduction attempt to portray? What don’t the authors address in this paper? What are the pros and cons of this approach versus related work?
      • Read the discussion section.
      • Read the related work section.
    • Take notes: Identify and note down any insights gleaned or references that look intriguing.
    • Answer your questions: If you have a list of questions you are trying to answer, did this paper help you answer any of them? If so:
      • Track this.
      • Verify (to the best of your ability) that the result you are relying on is correct. If you can’t tell, more background research, thinking, and hopefully, discussion with others, is required.
  • Pass #n for n \geq 3: These passes align pretty closely to Keshav’s “third pass”. This is where I try to understand the details of the paper, verify the correctness of any results of interest, try to identity where I agree or disagree with unstated assumptions, etc. But this is not a single pass for me and it’s almost never front to back:
    • If I had a specific question, e.g., if I wanted to verify a particular result or check my understanding of a particular concept, it’s helpful to pinpoint that part of the paper and only focus on the relevant parts.
    • A friend of mine from mathematics advises:
      • “If I’m looking to check a particular fact in a reference, I’ll try to start at the particular theorem and gradually expand my scope until everything I need is defined.”
      • “Otherwise if I’m generally interested in the paper, I’ll do multiple passes. First pass is literally flipping the pages to just see the layout and the headings, then progressively more detail on each pass. But also on each pass there’s a point where I give up on absorbing/understanding, and that point is further along on each pass.” (Emphasis mine.)

Guideline #3: Use a paper management software system that includes decent search functionality, take organized notes for papers you read in any depth, and consider your equipment choices.

Many of these systems have deeply disturbing privacy implications if you enable cloud features rather than just running locally, however. I don’t advise ignoring this issue entirely, because intellectual freedom thrives in the absence of surveillance.

But it’s very helpful to be able to efficiently search through a large set of papers you’ve spent some time reading in the past. And taking notes on each pass (either directly in the pdf, or in some other form) can help you return to a paper later, regain context quickly, and achieve a deeper understanding on your next pass more efficiently.

To that end, consider what medium and tools work best for your understanding:

  • Have paper and a writing implement handy to sketch out mathematics, figures, and questions. White boards are nice, but keep in mind that you eventually erase white boards. (You might need alcohol to do so, yes, but eventually.)
  • Experiment with different mediums. I used to be addicted to paper, and there are still no hand-held devices that have appropriate dimensions for paper-reading in my opinion, but there is an assortment of e-readers, tablets, and monitors to consider.
    • I avoid reading papers in depth on my computer; I just don’t absorb the content as easily as in printed form or on an e-reader device.
    • But when I do read on the computer—which is often the case for passes #0–2—I want the text to be as crisp as possible, so I invested in a capable monitor. I wouldn’t trade crisp text for anything.

Leaving Oregon

Perhaps absent a pandemic, I would have stayed in Portland. The city has amazing food and a decent partner dance scene. But I think it far more likely that I would have left earlier than I did. In the end, Portland is as much a city as I ever want to live in longterm. Nothing really compares to a country sky at night.

But staying felt responsible in a world in which I had the privilege to stay put and travelling meant putting others at risk. It took a combination of losing my apartment, soaring housing prices with very little availability, a series of difficult compromises in my plans for Bella, and generous help from many corners to get me back on the road.

A picture of a lake with the sun glancing off the surface. In the background, a forest basks in the warmth of the sun's final rays. In the foreground is a large swath of dried-out lakebed, dotted with the eerie remains of tree trunks.
A lovely spot to mark my escape from the city.
High desert with a mountain range in the distance. A sunny day with a thick layer of clouds over the mountains.
Eastern Oregon, right before a cold front prompted me to head south.

But the road is an uncomfortable place to be right now. I will be glad to reach family and a place to curl up for the winter.

When I left Oregon, the healthcare system was still struggling and at capacity. Many of the places I have passed through since have been the same. The reality of overwhelmed hospitals with severe staffing shortages is at odds with the unmasked faces and crowded businesses.

The divide isn’t political. It seems that we have come together to declare the absence of pandemic. Where might we be, had we instead come together as a community and faced the death and disability that still surrounds us?

A dirt road stretching toward distance mountains in the high desert or eastern Oregon.
Yes, that’s right. A road.

The Travelling Cryptographer is Back

But I’ve downsized. Considerably.

Meet Bella:

A black Ram 1500 Rebel.
Bella is a big black truck. A 2020 Ram 1500 Rebel that had been misfiled as a different model on the Dave Smith website. Only reason she hadn’t already sold as far as I can tell. The only other one I could find in the US was in North Carolina.

Bella was difficult to acquire. I got her back in September 2020, after determining the pandemic supply chains were a bit more reliable for outfitting a truck than a van, and after realizing that, well, I don’t actually much like vans, anyway. Half of that turned out to be true. Maybe vans will grow on me and I’ll have to eat all my words.

I debated going to Idaho to pick her up, but after being reassured that “We don’t bother with all that pandemic stuff over here”, I opted for delivery. The truck truck driver showed up, sized me up, and immediately asked “What’s a little lady like you gonna do with a big truck like this?” Enough said.

The first step was acquiring some kind of canopy. After finding an awesome local company that would help me out, it looked like the wait would be only a couple of months. Try ten.

Meet Bella with a Super Pacific hat on:

And we’ve gained 388 pounds.

You might think I’m being sarcastic, but Super Pacific actually is awesome. Great folks work there and they make a solid rooftop tent/canopy that so far seems to make a lot of people envious. They’ve also spent quite a few additional hours helping me modify the canopy. (But more on that later.) Supply chains really are just fucked. Emphasis on the present tense.

Because, you know, wishing and being tired doesn’t make the pandemic actually over.

Next stops: modifying the canopy, electrical, an awning adventure, and finally waving goodbye to Portland. Turns out moving to a place just before a global pandemic doesn’t suit.

Remembrance of Earth’s Past Trilogy

I finished the last book of Liu Cixin’s trilogy, Death’s End, a few weeks ago, and the first two last fall. So this review will be necessarily a bit more high-level and about my general impressions, because memory is faulty. Feel free to skip to the end for some quotes, though, because I have a habit of taking pictures to memorialize “oh yes, that’s really what it says” moments.

I can see why the series was so popular: it’s engaging, the sci-fi aspects are fun, and I found the pessimism underlying the imagined interactions with alien life forms and the supposed eventual collapse of the universe addictive.

But I can’t escape from the conclusion that Liu is sexist and appears unable to imagine characters of any depth. At first I thought that the sexism was not central to the author’s storyline, but Death’s End quickly disabused me of that notion. The entire plot seems to hinge on humanity becoming overly “effeminate” and making a catastrophic choice to designate a woman as the protector and ultimate decision-maker of the species. Taken as a whole, the downfall of humanity appears to rest on the misanthropy of one woman and the “maternal instincts” of another.

We are also treated to a series of absurdly flat relationships that fit an unimaginative, traditional dynamic that brings to mind stereotypes of 1950s America, oddly mismatched for the setting of a futuristic society in outer space.

We watch as the recipient of society’s misplaced faith is periodically woken up from near-constant hibernation to make decisions that have a disproportionate impact on the Milky Way’s population. She doesn’t so much have a story of her own: she is either unconscious, recovering her consciousness, briefly glimpsing other people living at various points in history and pining for a lost romantic opportunity, and then making snap decisions—mostly about humanity’s future rather than her own—before going back to hibernation. When death seems imminent, she even chooses to use a “sleep-aid machine” for her last space voyage. Because her preferred and habitual mode of existence doesn’t involve thinking, or much of anything at all, I guess.

Her romance is awkwardly entangled with the survival-of-humanity arc, both of which end on a very one-dimensional, uncompelling note. As a reader, you might be forgiven for concluding that this note is actually a constant throughout the story.

Selection of quotes from the books:

***

In Chen Xin’s subconscious, she was a protector, not a destroyer; she was a woman, not a warrior.

On why humanity falls.

***

… “Deterrence made a comfortable cradle, and as humanity napped inside, it regressed from an adult to a child.”

“Don’t you know that there are no more men on Earth?” someone from Gravity shouted.

On why humanity falls.

***

… the bodies of modern humans had changed considerably from the past. They were more flexible and agile compared to past generations, but were no longer adapted to boring, repetitious physical labor.

On why humanity can’t farm anymore.

***

No doubt this deeply melancholy man was very attractive to a woman’s eyes. But this didn’t worry him, because the man was so obviously consumed by his despair that nothing else had any meaning for him.

Because if man #2 had not been so consumed by his despair, the woman in question would certainly go for him, so man #1 had best worry in that case.

***

“I’m sorry, love. I’ve aged eight years,” Keiko Yamasuki said.

“Even so, you’re still a year younger than me,” he said as he looked her over. Time seemed to have left no mark on her body, but she looked pale and weak in the fog’s watery moonlight. In the fog and moonlight, she reminded him of that night in the bamboo grove in their yard in Japan. “Didn’t we agree that you would enter hibernation two years after me? Why have you waited all this time?”

Because when you’ve spent eight years working alone on your partnership’s mutual goals, definitely apologize for aging. Then be consoled because hey, at least you’re still a year younger than your no-longer-hibernating partner. But then explain why you chose to work an extra six years instead of preserving your body.

***

Chen Xin had the impression that many people here seemed to be from the Common Era, but soon realized she was wrong ….

Her impression was due to the fact that she saw some men who looked like the men she was used to.

The men who had disappeared during the Deterrence Era had returned. This was another age capable of producing men.

Everyone seemed to be in a hurry. This seemed to another swing of the pendulum: the leisure and comfort of the last age had disappeared, and it was once again a harried society. In this age, most people no longer belonged to the leisure class, but had to work for a living.

No comment.

***

AA and Chen Xin didn’t bother to watch more news. It was possible that, just like Cao Bin said, the Bunker World had approached paradise. They wanted to see what paradise looked like, but they didn’t dare look. If everything was heading toward ruination, the more beautiful it all was, the more pain they would suffer. In any event, it was a paradise that was collapsing in the terror of death.

Definitely go back to sleep instead.

***

 

Orders of Magnitude

As a full disclaimer, I’m terrible at arithmetic. Particularly if I think someone is analyzing my speed at manipulating numbers in my head. I think part of it is because I’ve had more than the average training in mathematics, and I think this is somehow supposed to make me a computer in the original sense of the word. I don’t have the attention span to have thrived in that particular profession.

As another disclaimer, even though I’m going to talk about COVID-19, I’m not claiming to be anything other than your typical armchair numbers person inspired by a global pandemic to ponder infectious disease. Except I guess it’s more like “standing desk” or “floor-sitting” numbers person, because I don’t own much furniture. But that’s another story.

All that to say, this article is inspiring a digression on orders of magnitude and the importance of context in interpreting numbers. The highlight is that the Governor of Oregon erroneously claimed a deal with Quest to buy 20,000 test kits (turns out the deal was for “just” 10,000):

Brown hastily announced a deal that wasn’t finalized and trumpeted an eye-popping number of tests that didn’t prove to be accurate.

Oregon, like much of the country, has a testing and tracing problem that it does not appear to be solving, which I think is what the author of the article really wanted to say. I’m not going to try for a precise definition of “eye-popping” here, but I do imagine the term would satisfy the following:

  • If x is an “eye-popping” number, than x/2 is also an eye-popping number.
  • If x is an “eye-popping” number, then x is probably at least an order of magnitude larger than 20,000.

For context, 20,000 is approximately the number of tests Oregon had already completed at the time of the article’s writing (April 8). This is not a particularly large number relative to Oregon’s population (~4.2 million) or relative to its geographic location in the context of this pandemic (sandwiched between our dear neighbors Washington and California, two of the first “hot spots” in the nation). Or relative to the length of time the disease has probably been spreading locally.

Which is to say, if Oregon had conducted 40,000 targeted tests as part of an early and ongoing test and trace program, I guess we wouldn’t be in bad shape here. Provided we had combined that with giving resources (e.g., food, basic sanitation supplies, housing) to those under quarantine.

But, together with the rest of the US, we didn’t. Are we even in catch up mode? I don’t have a beef with the gist of the above article, which goes on to point out that Oregon, at least, doesn’t seem to actually be ramping up its test and trace capabilities. Despite what the governor had claimed.

As the news devotes more and more time to antibody tests and visions of the recovered “reopening the economy”, it’s worth pointing out that even in the areas that are hardest hit, the group of people who have been infected with and recovered from COVID-19 is really quite small and we don’t yet know if the immune response is robust or lasting.

Put another away, most people are still susceptible to infection, and those who have already been infected and recovered may be susceptible to reinfection far sooner than we’d like. (And here I don’t mean a measly ‘most’, but more like oh, I don’t know, well over 99% of humanity.)

Oh, and let’s not forget that viruses mutate.

This Vox article is worth a read.

Unsupported Conclusions

Lately I feel as though I’m drowning in a sea of unsupported conclusions whenever I read the news. Today’s winner is this article from the New York Times, which includes the statement “Americans eat many more vegetables when meals are prepared for them in restaurants than when they cook for themselves”.

I’m not going to debate the underlying causes, motivations, or possible fixes for the disastrous affair that is our current food growth and distribution system, at least not in this post.

Instead I just want to point out that:

  1. The correspondence between (food bought by restaurants) and (food bought by restaurants that is eaten by humans) is not one-to-one. The restaurant system is not zero waste.
  2. You cannot draw a conclusion about how many vegetables Americans generally eat when cooking at home vs restaurants in the context of a global pandemic. For example, consider the added anxieties people are experiencing with respect to grocery shopping:
    • I’ve seen many debates over the extent to which it is necessary to sanitize food, particularly produce. I’ve seen all sorts of ridiculous advice floating around, the worst of which was probably the Michigan doctor who decided all produce needed to be washed with soap*. Other advice I’ve seen is to avoid fresh produce altogether, and instead opt for frozen fruits and vegetables.
    • Lots of advice is to limit the frequency of your grocery trips to once every two weeks. This is a design choice that inherently limits how much fresh produce you’ll buy.

By the way, has anyone seen any recent figures for vegetable/fruit purchasing and consumption by ordinary people? I haven’t. I can tell you that Misfits Market has a waitlist now, because so many new people have signed up for their service and they haven’t yet been able to scale up to meet demand. A similar service, Imperfect Foods has also expressed increased demand and corresponding delivery adjustments/delays.

I’ve also seen efforts on a small scale to increase CSA/buying club availabilities to help small farmers find alternative, local markets, and I look forward to seeing what results from these and similar experiments.

Additional point with respect to #2 brought up by a friend: Let’s not forget that people are capable of growing some portion of their own produce. Victory Gardens, anyone? Even in my tiny corner of the world, I’ve seen lots of first-time gardener posts in my neighborhood groups.

*Your stomach will not thank you for consuming soap.

Silence: In the Age of Noise

I do not follow modern day explorers, and so the name Erling Kagge meant nothing to me until I picked up and read his beautiful prose, punctuated by art, in Silence: In the Age of Noise.

Kagge’s book is a reminder that observation is key to living well, and silence is key to observation. That is not to say silence in the most literal sense, necessarily; Kagge reconciles the silence of isolation, of trekking to the far reaches of the Earth, and the silence of the city, the silence of an individual in a crowd.  Silence of a sort can be found in the midst of noise, if you are willing to observe the moment.

But to say that literal silence is unimportant would be to misrepresent the book entirely. Certainly I have felt inundated by the traffic, the congestion, the constant pressing of others onto personal space, the ubiquitous noise of televisions, radios, and the whispers of headphones deafening the wearers day by day, the periodic pinging of a phone, a watch, an impatient microwave. I have often felt that it is impossible to be self-aware without periodic silence. With noise it is too easy to lose yourself in the cacophony of stimuli, to never have to face the innermost workings of your mind, to never slow down enough to experience poetry.

I am grateful to have discovered Rolf Jacobsen, whose poem The Silence that Follows is excerpted as below; it is well worth reading the whole poem:

The silence that lives in the grass
on the underside of each blade
and in the blue space between the stones.

Kagge also reminds us of the classist nature of society through the lens of noise. We live in age of noise and few people have the luxury to escape to the quiet of remote nature. I once lived in my tiny trailer by a railroad and the popular exit of a highway; the noise could suffocate you, if you let it. The freight trains came at night and enveloped your body in the vibrations of their passing. It was the worst area of town, the part of town people tell you to avoid because the air quality is atrocious, the homeless encampment (the regulars almost exclusively composed of veterans) is next door, and your neighbors are bike thieves and drug dealers. But I was still lucky, because my neighbors were also hard-working immigrant families who gave me fresh eggs, ex-drug dealers who had decided to be responsible single parents, itinerant ER nurses, and young men who dreamed of being physical therapists because, they said, they knew so many people in physical pain who could not afford treatment. And because the time I looked sad and weary, carrying a hiker’s backpack and a guitar in the relentless heat, the veterans on the corner asked me if I needed help. And because my commute offered a sudden escape from the noise, an arboretum with a water body that passed as a creek. I walked to work sometimes because to bike meant less silence.

And when I left my job, I was even luckier, because I was privileged to have unending days of silence on the road.

 

 

 

Food Fights, Barbecuing, and Community

I rarely read one book at a time—and in an attempt to avoid the sunk cost fallacy I sometimes opt out of finishing books—so I thought perhaps I’d review books as I am reading them.A cat on the book!

Food Fights and Culture Wars: A Secret History of Taste, by Tom Nealon, is a delightful book by any measure, but most of all from the viewpoint of aesthetics. I can’t imagine this book was at all cheap to produce, as it abounds with hiqh-quality color reproductions of art and historical advertisements, many of them courtesy of the British Library. The writing is a bit tongue-in-cheek, but enjoyably so, at least if you don’t mind irreverent references to cannibalism of “inadvertently delicious” corn-fed Aztec lower classes.

My favorite chapter from the book is on the history of barbecuing; here the tone diverges sharply from humorous to political. The contrast between traditional barbecuing (community affairs involving whole animals  and fire pits) and the commercialized, modern version of restaurant take-out and backyard grilling, is placed in the context of dwindling freedom of assembly. Nealon points out that in many areas of the United States, at least, fire pits are forbidden or at least require a permit, and are generally frowned upon at public parks, although apparently—to my surprise, anyway—barbecuing used to be considered acceptable on public land.

Personally I hadn’t realized the distinction between “barbecuing” and “grilling”, or considered how neatly the backyard grill fits into a capitalist paradigm of individualized consumption, a co-opting of a once inexpensive and widely accessible form of community-building and social discourse. To the extent that barbecues can still be held today, they likely require the privilege of land ownership and the privilege of a permit. Which is to say, if you have the advantage of a place and permission to barbecue, you should pay heed to Nealon’s concluding paragraph:

It’s not the world’s biggest surprise that corporations and governments have been destroying barbecue culture to further nefarious agendas. No, it is as it should be: barbecue has been forced out of range. Attempts to co-opt and commodify only prove the point: humanity and barbecue need one another. It is up to us to exercise some self-control the next time we have a hankering for some ribs: to build a fire instead of ordering in, to dig a pit, invite our friends, acquaintances, and one or two enemies, and stir up a sauce, a marinade, and a rub. As our lives become ever more full of flickering lights, ephemeral sounds, and unmoored notions dancing in our peripheral vision, it becomes more important than ever to take a moment, a breath, a mouthful. Every day we are enshrouded in a digital chaos so complete that is has become a sort of order, and I submit that what it screams for, more than anything else, is a properly made pulled-pork sandwich. All you vegetarians out there, don’t think you are off the hook: catch yourself some root vegetables, perhaps a great woolly radish, a tremendous turnip or an outrageous rutabaga. Find yourself a pumpkin the size of a space hopper and barbecue some soup — but take it slowly. S-l-o-w-l-y.

I’d probably add that you should invite some random passerby, too. And perhaps consider barbecue-as-protest.