Ready or not, time to grapple with e-memory

Just because Gordon Bell and Jim Gemmell are way out there on the nerd spectrum, don't ignore what they have to say in their new book, "Total Recall."

The Microsoft researchers obsessively record e-mails, photos, videos, phone calls, health records, financial transactions, Web site visits, and everything else they can in an attempt to electronically compensate for the fallibility of their own biological memories. Before you recoil at the prospect of letting your own life become this digitally augmented, though, consider that it will be whether you want it or not.

Gordon Bell

(Credit: Gordon Bell)

"Total Recall," which goes on sale Thursday, is a reasonable and general introduction to the idea that you might want to digitally record much of your life. Chiefly, the book exists to encourage people to take the plunge, but it's also got practical advice about doing so, some warnings about unpleasant possible side effects, and even 10 related start-ups Bell says he'd invest in.

The book is a useful, accessible work from people who've already examined the issue in technical detail. I believe the pair overpromise on the near-term benefits of e-memory and understate some of its difficulties, but they don't try to pretend it'll be universally beneficial.

Bell and Gemmell have real chops in the area. Bell, who started his recording effort in 1998, designed seminal computers for Digital Equipment Corp., established a supercomputing prize in his own name, co-founded the Computer History Museum, and joined Microsoft Research in 1995. Gemmell works on next-generation search and personal information storage, and with Bell and coworker Roger Leuder helped create MyLifeBits software to try to give people a handle on their electronic records. They're the kind of folks who can be found with a Microsoft SenseCam slung around their necks to capture images, sound, and other data as they live their lives.

"We regard it as a memory supplement or surrogate. We're trying to offload biomemories," Bell said. Though plenty of people like to share their lives digitally, the authors see their effort as very private.

So why should the average person listen to them? Because they're right about one thing: all this information likely will be recorded one way or the other as sensors proliferate and digital storage gets ever cheaper and more capacious. It's best to have your own copy--and it's likely it will in fact prove useful.

The authors have high hopes: "higher productivity, more vitality and longer life spans, deeper and wider knowledge of our world and ways to accomplish things in it."

Jim Gemmell

(Credit: Miriam Gemmell)

Yeah, right
Still, it's a hard sell.

I doubt most people will embrace what Gemmell and Bell call e-memory with much enthusiasm. Most folks see filing as a necessary evil at best, and e-memory dramatically increases the amount necessary. What file system should you pick for your photos, GPS location logs, energy consumption measurements, and blood pressure records? What backup strategy? What privacy settings? What computer hardware, software, and online services?

I've been stewing over some of these questions for a while--my recent effort to go mostly paperless forced the issue--but my conversation with the authors still was thought-provoking.

History by some definitions began when people started recording things, and the Total Recall era of Bell and Gemmell has the potential to bring as radical a shift in human behavior as the arrival of books.

If this shift is to come to pass, a key factor will be extracting something useful out of all the data you collect. "The root of the problem is to get the computer to first record it and store it and somehow be able to act on it," Bell said.

The obvious mechanism is search, of course, but Gemmell and Bell also advocate wall-hanging screen savers that constantly replay parts of your life in the background. Another idea of theirs got me thinking about things more holistically.

Specifically, there's the idea of correlation: collecting data in one area could help you find or use data in another. For example, a log of your position kept through a GPS system in your phone could help you retrieve a message you knew you sent while on a particular trip. Or in a more sci-fi scenario, your logs of movies and books read, cross-referenced with your heart rate and blood pressure measurements, could help you decide what sorts of movie to watch.

Then there's the idea of prediction. A sufficiently smart system processing your collected data could not just remind you of simple things such as birthdays, but also anticipate more complicated advice--time to get a check-up or spend more time with friends, perhaps.

MyLifeBits can be used to catalog and search personal archives.

(Credit: Dutton Books)

Troubles ahead?
Here's where I see storm clouds on the e-memory horizon, though.

First, if it's easy for me to store everything and perform sophisticated correlation analysis, it's easy for scheming governments or profit-hungry corporations to do the same. With the arrival of online behavior monitoring and omnipresent surveillance cameras, it's harder to find a corner of the world to claim as your own.

One answer from Gemmell and Bell: "Big Brother, meet Little Brother." Your personal records can give you an alibi proving your innocence in some matter. But that's only OK as long as you happen to live someplace where the rule of law has some teeth and the authorities aren't abusing their power.

And if everybody walks around with recording devices running around the clock, the government may be the least of your concerns for privacy. Politicians today are still adapting to the YouTube era when every moment is on the record, but we're headed toward an era when it applies to us all.

Perhaps that will deter crime on the assumption that people behave better when they know they're being watched. But perhaps also some white lies and imperfect memories help us all get along. Can we really handle the unvarnished truth about our friends and coworkers, much less ourselves?

The authors believe it's time to adapt to a world in which truth and honesty are harder to overlook, and that social protocols will adapt perforce. For those moments when you want some Nixonian plausible deniability, perhaps protocols will emerge to hold a conversation that's logged only in people's neurons.

Bell and Gemmell expect new conventions for asking others' permission to record. But if data capture is as easy, ubiquitous, and unobtrusive as "Total Recall" suggests, it looks like it would be wiser to assume everything is being recorded already.

Fragmented records
From a technological standpoint, I worry about data being fragmented. When Bell started his work, he was a fan of centralized data stored on your own hard drives. But now we have the cloud, too--online sites not just for backup but increasingly the central repository of your data held in some service.

Companies such as Yahoo, Facebook, and Google have plenty of incentives to set up services that house precious data such as photos, communications with friends, and documents--but they may not want to share that openly with you or some company that offers e-memory services on your behalf. And even if they do, there are practical barriers aplenty.

Perfectly preserved ones and zeros can nevertheless decay with age, locked in obsolete formats. The researchers advocate periodic backup into "golden" formats such as PDF or JPEG whose widespread use will protect against obsolescence. Ironically, Bell himself illustrates the importance of the problem: the Microsoft Money software he touts for tracking his own financial transactions is being phased out.

Data ownership is another complicated matter. As Bell and Gemmell observe, parents today already electronically monitor their children closely and exhaustively document their lives. I wonder if or when the resulting information should become the child's property.

And in the most sci-fi section of the book, the authors ponder digital immortality, the possibility of conversing with a digital avatar representing the deceased. "How I wish I had even a tenth of my grandfather's life," Bell says--but how much of one's life should belong to one's heirs?

The authors and I aren't the only ones to have pondered the ideas of e-memory and extrapolated into the future. They cite various science fiction books that have bearing on their ideas, for example.

But it was politician and presidential progenitor Vannevar Bush whose "memex" provided a template for the e-memory work in a 1945 essay, in The Atlantic Monthly, "As We May Think." His idea: "A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory."

It's happening
Even though I'm not convinced everybody will eagerly embrace e-memory, I am convinced that it figures centrally in our lives. I have seen the power of Gmail's archive and my own digital photo archive. And as it gets easier to do record everything, we will find a way to make it useful.

Many years ago, I made T-shirt presents for my family that used the highest-resolution scans I could make of some graphics. At the time, I was astounded at how much space each image took up--20MB or more. Over years of hard drive tidying efforts, I considered deleting the files many times, but they had some sentimental value and I put off the decision each time.

Now I'm glad I hung onto them. The T-shirts have all worn out, but I still have the artwork--and 20MB is a piddling amount of storage space. For comparison, each of the thousands of photos I've taken with my present camera today is closer to 30MB.

At the same time, I find myself e-mailing myself notes, recording voice memos, keeping my GPS logs of excursions, and taking snapshots of the signs and maps at trailheads. Without even trying hard, I'm assembling my personal life's digital equivalent of everything from Library of Alexandria to the trash heaps archaeologists delight in excavating.

I'm a ways out there on the nerd spectrum, too. But with ordinary folks carrying camera-enabled smartphones, e-mailing their way through work, and socializing through status updates, brace yourself for e-memory to arrive in your life.

[42istheanswer]

I find this activity fascinating. On the one hand it could be useful for memory supplementation. On the other hand, I can imagine how much time has to allocated to keep this up. In the end, it's useless. We should just be organic humans and live life as it was intended to be. Connecting with nature and developing ourselves in preparation for the next phase. I don't think living in the digital world will accomplish that. I guess when you're M$ and you have the kind of money they do, you can dabble in various useless endeavours.

[Shankland]

What exactly does "live life as it was intended to be" mean? Does that include surfing the Net, driving, trading goods globally, taking antibiotics, and honeymooning at Niagara Falls, just to pick a few random activities our ancestors probably would have had a hard time conceiving of? Even without e-memory available on my PC or, in the vision of Ray Kurzweil, directly interfaced with my own brain, it seems to me that we're already far away from a tribe of hunter-gatherers, from Mesopotamian farmers, or from 1950s suburbanites. I'm a big fan of connecting with nature, but it's pretty hard to draw the line between what's natural and what's artificial.

[cvaldes1831]

"I'm a big fan of connecting with nature, but it's pretty hard to draw the line between what's natural and what's artificial."

Riding my stationary bike in my condo: artificial. Jogging through the open space preserve, knowing that there's probably a mountain lion within a couple of miles of where I am: natural.

Surfing the Internet: artificial. Surfing Linda Mar, knowing that Ol' Whitey is silently lurking nearby (they don't call it the Red Triangle for nothing): natural.

Not too difficult.

[Shankland]

@cvaldes1831 Did you drive to your surf or running spots? Did you go barefoot when you were running? Did you grow your own food to supply your energy? Technology is everywhere.

[cvaldes1831]

I see that you enjoy debating semantics, something I do not, so I will bow out of this discussion.

Best of luck.

[trewbux]

"live life as it was intended to be" -- as who intended it to be? Intention is something that comes from humans - there is no intention outside of intelligent living things. When you look at the statement in this light you see that all we can do is live life as WE intend it to be. So you can pick some arbitrary point in our history (and technology has been a part of humanity for tens of thousands of years), and say that this is what I intend my life to be. Or you can embrace the fact that technology is really a natural extension of our biological evolution, and discover what it has to offer.

[jaguar717]

What file system should I pick for my GPS location logs and "energy consumption measurements"?

How about none whatsoever. I can back up pictures and documents to a hard drive; done and done.

[blusky08]

Yes, it's hard to define what "live life as it was intended to be" means. However, we are truly ancient biological systems with many ingrained instincts (whether we want to face it or not). This is a transition that may push us over the edge as a species. Humans beings aren't capable of being held to the standard that constant surveillance would require. And, has technology really made us happier overall? It's suppose to simplify life, but does it really? Take staying connected via email and mobile phone constantly--tech junkie or not, IMO life was not only simpler but also happier (more "human" whatever that means) when life was focused on here and now rather than the next data transmission.

[ScottStein8]

I've been obsessed with this subject for years- will have to read the book. Thanks for the article.

[cloudmatt]

1984 nuff said

[fokkwp]

Actually, we are getting ever farther from the possibility of digitally recording all that happens in your life for one simple reason: an increasing part of your life is the digital life of others, which expands exponentially. The time you spend on Skype, Facebook, Twitter, CNN News, Youtube, Netfilx, Flickr, text messaging, - all that is part of your life, integrally so, and it is all going from a few grainy images to High Def. You'll never record all this now. Whatever your digital footprint is, the sum of the digital footprints you engage with on a daily basis will be far greater.

[Eddie-c]

When is the ability to offload/ditch unwanted memories, per Johnny Mnemonic, coming? ;)
(And I'm sure that some people would think it would be 'cool' to upload a couple of TB of MP3s into their head) ;)

[SergeM256]

Why would anybody want to collect and preserve all this information? Who is going to use/read/view this information? It is OK to save a couple of dozen vacation photos but I realized a long ago that if there is a couple of hundred photos on a single subject it gets to time-consuming to view them and they never get viewed more then once (sometimes not even once).
It reminds me when I was about 10 years old I had this idea that I have to start diary to make it easier for historians in a future study a life of a great man that I was meant to be.

[Shankland]

I agree to an extent. I winnow down my photos, and not just obvious duds, and I do delete some e-mail that conceivably might be useful some day. In the e-memory vision of Total Recall, though, they advocate keeping just about everything. Their reasoning is that you never know what might be useful either directly or as part of a collective body of data or as a means to getting to some other information you might want. More importantly, they argue that computers will get good enough to sift through the dreck to find the good stuff for you.

[Hokulea]

Following in my father's footsteps, I'm one of those guys that has a hard time throwing away anything that might possibly be useful. As you may imagine, the result is a collection of things that have long outlived their usefulness. With the passage of time, I've become much more discerning regarding what to keep and what to throw away. Of course, eBay is also an option. That way I can pass my junk on to others who are even more retentive, and hopefully at least cover shipping costs.

As I too attempted to go paperless, several pitfalls revealed themselves. One result is that my organizational strategies have continually evolved. Another is that the more I hold on to, the more storage space I need as well as even more for backups. Fortunately, the cost of storage has come down dramatically in the last few years. A 1 TB drive today costs about the same as a 250 GB drive did a few years ago. I used to burn backups to DVD's, but the cost per GB now favors hard drives. With four external drives and four internal drives holding several TB's, an organizational strategy becomes key. I have found what works for me through trial and error, mostly error.

One thing I will not do, is trust my data to a cloud solution. My concerns relate to both security and privacy. I also have concerns regarding where the physical server is located. I don't want my data stored overseas where there may be less legal protection. Another issue is deleting data from a cloud server. While a user may not be able to access deleted data, that doesn't necessarily mean it's still not being retained somewhere. I'm just not comfortable relinquishing control over my data to a second or third party. My experience has been that contractual agreements can change over time, often without notice. When they do, it's rarely in my favor. Privacy policies can also change, again often without notification. Privacy policies also change, again often without notification, and they trend towards less not more protection.

My mother put together a scrapbook that details my life as I grew up. In it are a couple locks of my hair, faded photos, my good report cards from grade-school, as well as some really bad attempts at art. Leafing through it gives me a tangible connection to my childhood, and to my mother, that the same things displayed on a computer monitor could never do. Some things are still better represented in their physical form instead of as a collection of bits and bytes.

[No Man]

Stephen, have you seen the pilot episode of Battlestar Galactica: Caprica? It deals with this very issue and then goes one step further. Specifically, with a sufficiently complete digital footprint and sufficiently complex AI software, could you recreate a person's consciousness? If you knew every event from their lives, how they reacted to every single decision they ever faced, every facet of their personality (both public and private), etc could you make an AI so indistinguishable from the original person that it would be a perfect duplicate? Fascinating concept, though probably more than a few years away.

[EvanSei]

computers already store a good amount of what we know how I see it all they would be doing is putting all of this in one spot (besides the internet that is) and it really is a good idea. that is until bad people decide they want your information. If those people can get in they have access to everything about you, EVERYTHING, just a bit unnerving. even now if somebody gets into your computer they don't know everything about you, allot but not everything. Most digital technology is far from secure and I think I will hold off on transferring my complete life onto them until that changes, oh and data loss could be a problem. What happens when the thing crashes, no matter how good the back up some information is lost.

[0ri0n]

I trust anyone else to safeguard the integrity of my data as much as I trust politicians to work without a political strategist who manipulates facts in their favor, lawyers who will exploit or sensationalize upon specific information to generate a bias in their favor, insurance companies who use every viable tidbit of information to make you look like an insurance risk, or corporate marketing companies looking to capitalize on your personal information.

Until humanity has a level of integrity I can rely upon, I see recording my life as more a intimate violation and therefore no reason to trust anyone with my information but my self.

The personal aspects of my life are highly valuable, or you would not have so many companies wanting to record it, analyze it, share it. But is any of use getting compensated for it? Or are we just lead to believe it will 'all make life so much easier if we just cooperate in its submission'?

[LifeGraph]

I think Gordon and Jim's work is fantastic. I conceptualized an e-memory like object I call LifeGraphs. My aim is to persuade traders of goods and services to be ontologically compliant to a global data model of their products that have 3D geometry (surface and volumetric). The goal is for legal and natural persons to have the capability of a mobile mirror world that is composed of digitally signed 3D scenegraphs representing the items they own and/or purchased from their birth or incorporation to their postmortem or disestablishment. The technology stack using many XML based solutions like the ISO ratified X3D and Human Animation standards. The idea came from a Leading Edge Grant that my employer, Computer Sciences Corporation, awarded me in 2007. Improving the links between the chains of supply, value and customers/consumers with computer-aided design artifacts repurposed for virtual/hybrid/augmented/mixed reality engines that are integrated with behavior pattern algorthims for all types of common life parameters is my next step. I hope to meet Gordon and Jim to discuss the roadmap to their vision and mine.