The ignominious failure of Tracking All the Things
And my close call with fixing [REDACTED]!
Project Try Everything is two years old … and I am now done, as in fed up, with logging the symptoms of my mysterious affliction, or anything that could possibly be causing or triggering them.
My mighty Spreadsheet of Misery sits untouched, as dusty as a piece of exercise equipment in the garage. Not only have I dropped it, but I’ve also avoided saying anything about it for a while — because it is a big drag, and I’ve had a rough winter. There would have been a lot of symptoms to log, had I still been logging anything.
For about eighteen months I meticulously quantified my miseries, hoping to wring a single useful insight out of it. But I failed to identify even one robust correlation between any symptom and any modifiable risk factor.
I gave up when my most promising possible correlation — with a particularly awful symptom — crashed and burned.
Logging failed me even when it seemed like it had identified a cure for [REDACTED]!
For about three months last summer I thought I had truly identified the cause or major trigger of one of my worst-ever symptoms, a problem that is rather … well … let’s just say “[REDACTED]”!
That symptom has been extremely rough on morale.
An early draft of this post was my victory dance. I truly believed that I had beaten [REDACTED]! I seemed like I’d learned something truly important from my logging, and that made it okay to “too much information” about a cringe-inducing symptom. I was willing to write a TMI post if the awfulness was in the rear view mirror.
But then the [REDACTED] came back and [REDACTED] so much 🤬 [REDACTED]! That setback destroyed everything I thought I’d learned about my [REDACTED], and I faced this double whammy disappointment:
The misery of knowing that I am stuck with [REDACTED] indefinitely. It just seems unbeatable, like all my other symptoms to date.
The heartbreak of knowing that that all my logging — probably hundreds of hours over that year and a half — was all in vain. And so I quit.
And that strikes right at the heart of Project Try Everything. It has been more than five months since I last posted, a gap I never imagined allowing. For a long time, I didn't know what to say. I had failed at logging and blogging!
But so it goes with chronic illness.
So maybe don’t bother logging your symptoms
In the spring of 2022, at the beginning of this project, I wrote:
Tracking and logging is the science-y heart of my self-rescue project. I wouldn’t call it “science,” exactly, but the idea is to approach the problem with something like scientific rigour.
I have many theories about what might be wrong with me and what might hurt or help my symptoms — even if it has nothing to do with the cause — and I cannot test those ideas if I don’t have data. Identifying correlations between symptoms and the many factors that affect them can be extremely difficult.
It knew it would be difficult, but I assumed that it was just a matter of time and diligence before my investment bore fruit. I thought that my experience would be the basis for advice that I would give to readers and friends and family for the rest of my life: if you're desperate, start logging!
Ah, sweet summer child.
Your mileage may vary, of course. Perhaps some people can learn something from logging — simpler cases, fewer symptoms, more likely triggers.
But the return on my investment was pathetic. And I think that it’s overall because just because the problem is just too hard. The odds of learning anything that matters are poor at best.
And so my advice to most people is… beware of the risk of wasting your time. 🙁 Consider using that time and energy for anything else.
Why logging probably mostly can’t identify the causes or triggers of symptoms
There are too many variables, and measuring and quantifying and correlating them is too messy.
There is a stupid amount of noise; the signal is next to impossible to detect.
The effort and expertise required to solve these problems is beyond most of us. Hell, most people don’t even know much about how to use a spreadsheet! And having some spreadsheet skillz seems like the bare minimum. You probably also need to apply some non-trivial math and stats.
And so…
If a correlation between a behaviour and a symptom is strong enough to be identifiable… you probably already knew it. Red wine triggers your migraines? No logging needed!
But, if you didn't already know about (or strongly suspect) such a correlation … good luck identify it with logging! The subtle ones are extremely elusive.
Could there be a useful intermediate difficulty level? I held on to this hope for a while, and it’s the hope that kept me logging. For example, I hoped that food→symptom links might be in a sweet spot: tricky enough that you can't spot them withoutlogging, but still clear enough that logging can do the job.
A wild goose chase for a diet link was my biggest failure (and the nail in the coffin)
It seemed so clear! Several times in a row I saw that nasty [REDACTED] come and go in sync with the suspected trigger: artificial sweeteners, specifically Sugar Twin, a Canadian brand of "cyclamate," the least potent of the common artificial sweeteners.
I knew all too well that such correlations can easily be bogus. It’s a basic truth about scientific research that correlations are often spurious, and I’d already had several experiences with patterns that disappeared like a mirage shortly after noticing them.
And so I methodically, rigorously repeated my experiment several times — which is tedious! It took many weeks! But I wanted a "large sample size," so that I could be sure that the correlation wasn't just a fluke.
It sure didn't look like a fluke. The trigger and the [REDACTED] appeared and disappeared together exactly like one was causing the other. As Edward Tufte wrote, “Correlation is not be causation, but it sure is a hint.”
In the end I had at least three good, clean transitions between consuming and not consuming artificial sweeteners, three that were less well-defined but "probably" good enough, and another three that were probably confounded.
Roughly. I could look it up, but honestly I don’t think I ever want to see that spreadsheet again.
Confounder example 1: effect before cause?
In one transition, the [REDACTED] came back a day before I started using the artificial sweetener again! Fishy! That ain't right! Effects are supposed to follow causes, right? Pretty sure.
But there was always some volatility in the symptom, the odd day where [REDACTED] would manifest in the middle of an otherwise healthy patch, like a lone cloud passing in front of the sun on a beautiful day, but ickier. So I chalked this little data glitch up to a coincidence.
Or maybe a logging error. Maybe I just got the day wrong. There were plenty of occasions when I tried to backfill a day or two of logging from memory. There was often uncertainty about exactly when something happened, or how bad it was. This is another reason logging effectively is hard.
Confounder example B: data pollution!
I had to throw out at least half my data on this artificial sweetener thing because I discovered, weeks into the experiment, that I had been consuming another artificial sweetener. Oops. It was a different type, and the dosage was low, but still… confounding! There was no way to trust that data.
And consider how easily I could have missed that altogether. I found my mistake by accident. How many mistakes did I make that I never figured out?
Pattern? What pattern? No pattern here
Even after adjusting the data — half-assedly "controlling for" my confounders — I still thought I had smoking gun evidence that I really was being plagued by [REDACTED] only when consuming artificial sweeteners on a daily basis … and that I could therefore banish the symptom as easily as using a teaspon of sugar in my coffee instead of Sugar Twin.
So I tried to banish it. I stopped using Sugar Twin.
And the symptom did not go away like it was supposed to.
And so I started eating it again … and then symptom went away.
And that is when I gave up on logging.
The pattern was gone, and I would never see it again.
From the moment that I decided there was a clear, strong link between cyclamate and my [REDACTED] symptom … I never saw the correlation again. It has been months, and I have casually eliminated cyclamate several more times for a few days or a couple weeks at a time, and … yeah, the correlation is gone.
Right now I am consuming Sugar Twin daily, and have been for weeks, while enjoying the longest break from the symptom that I’ve had in about three years. It’s just gone at the moment. No more [REDACTED]! 🙄
But it will be back. And when it comes back, it won't make a lick of sense more than it ever has.
P.S. Someone's going to ask, so…
Yes, I have tried probiotics.
Yes, I have been to a medical specialist about [REDACTED], with the most predictable result in history of predictable results: the doctor was also stumped! "Have you been under an unusual amount of stress lately?" Yeah, because my [REDACTED] is constantly [REDACTED]!
Arghh the tedium of logging, only to have it not bear fruit. That sucks. For what it’s worth, I used to log symptoms constantly too (one part of that was for diagnosis of a hormonal disorder that could actually be tracked, and could only be diagnosed by tracking, but the specific symptoms were meaningless). Since joining my imperfect but useful CFS/long covid/fibro/related conditions recovery programme I have enjoyed their paradigm of not focusing on the symptoms at all, but rather focusing on health (health behaviours and health progress). From 20 years of practice their experience is that tracking symptoms doesn’t get you anywhere and the specific symptoms are often meaningless; all of them represent a dysfunctional and dysregulated system overall. It was a relief to let go of paying attention to them, because they were invariably bleak, and paying attention didn’t seem to change anything except my mood, which got lower! (My mum always says “where attention goes, energy flows”, and it always used to annoy me but it really proved true in this context.) And I have improved a lot over time, a rising tide of health easing all kinds of unrelated-seeming symptoms, so that is reinforcing as well. Obviously it was not just the release of focus on symptoms that has done that, but it was a necessary piece.
You know far more than me on science etc; I don’t know how relatable any of my experience would be to yours, of course! I am so wary of telling people stuff they already know or aren’t interested in 😄
Very sorry nothing was revealed with your meticulous tracking. I work with post-concussion patients and one of the general recommendations includes tracking symptoms along with weather, hydration, etc., to identify patterns. Out of all the patients I've worked with I think this was helpful for exactly one person. I'm careful not to suggest this across the board now. For some people it just is too frustrating when no patterns are found.
Then I did it to myself! I had strange long-Covid symptoms and naively thought if I just tracked enough I could get ahead of the symptoms and prevent them before they happened. I did it for about six months before accepting it was going nowhere.
Brains, bodies, and the environment they interact in are just too complex. And we don't know enough about what they're doing from day to day to find patterns. Like you said, some can be very obvious and repeatable. Red wine = migraine for me. Always will. But gin and tonic only= migraines sometimes. And it doesn't correlate with my monthly cycle like I thought it would. But I got tired of playing roulette so I don't drink anymore.