A Month With Generative AI (2022)

In August 2022, I found myself preoccupied with two things: generative AI, and Brian Eno’s diary from 1995 that was later published as a book—A Year With Swollen Appendices. I combined these two preoccupations into a project that swallowed my month: to “write something” about generative models and what they’d make possible, and to write a diary of the process behind the scenes. The “something” turned out to be “Toward Tools for Imagination,” posted on September 6, 2022. The diary stayed tucked away on Google Drive, shared with exactly four people. I returned to the process diary recently and decided there was no reason not to share more broadly, so here it is.

August 8, 2022

9:22am

This is the start of my generativity journal. Something that stuck with me from reading the Multiverse diary was the power of formatting. As a teenager, whenever I started a new journal (in a Word doc), I would spend ages playing around with what typeface would mark the new chapter best. Today, I’m trying to return to that some, starting with a very pale aquamarine background and bluey-purple typewriter text.

Something I know about myself is that setting timers works better than it should. What I mean: I would think that setting a timer for twenty minutes to “explore” or “read” would feel restrictive and absurd, like setting a five-minute-and-thirty-three-second timer to “enjoy myself.” (No more, no less.) But over the years, it’s the technique that’s stuck. Not quite Pomodoro, because I do twenty minutes and not twenty-five; it’s a quirk of this pattern’s origins in college, when I was working on my senior thesis about the 1895 Atlanta Cotton States and International Exposition. (Or did I develop it earlier? Maybe my old journals hold a clue.) With a precious wide-open day today, I want to make the most of it—and I think what will help is setting timers for twenty-minute stints, or sprints, and expecting myself to complete two of those in each clock hour. This gives me some wiggle room (I could take a twenty-minute break in an hour if I really needed) while still keeping the pressure on. I crave the pressure that will lead to breakthroughs, but it’s hard to find on my own. Timers can help.

(Nine minutes and twenty-two seconds left on this twenty-minute timer…)

I’m listening to Brian Eno’s Ambient 1: Music for Airports on the speaker in the guest room at home. It’s bright out—the green leaves seem to glow. I got into Brian Eno after spontaneously visiting The Interval with my husband Erik one day not too long ago, the bar on the ground floor of The Long Now Foundation. It was a magical half-hour that I hope I’ll remember forever. We’ve been back many times since; last week alone, I went twice. Over the bar, there hangs a piece of art by Brian Eno, and I remember seeing that and thinking “I didn’t know he made visual art.” Soon afterward, I searched to see if there were any books I could read about Brian Eno, and sure enough, his 1995 diary was published—in fact, the 25th anniversary edition came out just recently. How I loved reading that diary and seeing the truth of how small fascinations can spiral, how every artist needs (and struggles to create) hermit mode for themselves (more the more successful they are), how everyone’s a little petty in general and at the same time loving toward their own children. And I think there’s plenty for me in Eno’s exploration of generative music (is that what he calls it?)—algorithms that produce listenable ambient melodies. Way before large language models, there was this fascination…but it’s somehow still remained fringe, at least in the popular imagination. (Or my imagination?) Is it because it challenges our notions of creativity? It would be nice if that was it because it would make for a nice essay, but I’m not sure it’s the truth of it. I’ll just have to see.

To explore: the relationship between hermit mode and goblin mode.

My loose plan for today is to join the Midjourney Discord and start playing around with it. To what end? I don’t know! But I have to play with things to understand them, and it’s hard to find time to play. (Except by setting twenty-minute timers.) I burnt the first twenty minutes of this clock hour on poking around and procrastinating, so I think I’ll need to do these sprints back to back. Maybe I can at least sign up for the Discord now to get myself to a good place for the next sprint. And switch laptops; the other one seems somewhat revived.

Time’s up!

9:46am

A few minutes into my experience on Midjourney, some notes:

• I went straight for the #philosophy channel—of course I did. Makes me nervous to read because of the norms and tension inherent in online forums. Interesting to return to that energy, too—a big background to my college experience.

• Yveldy says “Well… it frequently create imaginary light sources (Like invisible light sources in games), fails to change perspective of objects, sucks at silhouette design etc they’re not exactly positives” re: the artifacts of AI art. Reminds me of Robin Sloan’s piece about the texture of various models and how he’s stuck with one for a long, long time.

• Reminder to self that I wanted to try Sudowrite today, too.

• For an essay if I write it: remembering the GPT-3 brainstorming session toward the end of the Breaker days.

• Kraemahz says: “Post-AI art as a form may focus on e.g. clashing colors that previous art theories stated were bad, just because those colors never show up in AI art.”

• The #prompt-craft channel!! One of my favorite observations (that I didn’t come up with; I think it might have come from that GPT-3 brainstorming session) is that there will be a new career of “Prompt Designer”—people who know how to get the most out of models by crafting the right prompt.

• Something to read, linked in a Midjourney channel: https://mordunkus.medium.com/the-artists-role-in-ai-creation-934d3cbca2b6

• Another thing to read, linked in a Midjourney channel: https://github.com/willwulfken/MidJourney-Styles-and-Keywords-Reference/blob/main/Pages/MJ_V3/Style_Pages/Themes_and_Design_Styles.md

• Kneejerk question from the job I’m in: what is Midjourney’s funding history?

• Pantalaimon in a Midjourney channel: “They did do that with this version, and they are probably going to allow people to train it to a style they like. The trouble is that sometimes you want it to give you left-field ideas, which it will do less and less of, if they keep retraining it on user-rated-output.”

• What a prompt: “on a dark desert highway, cool wind in my hair, Warm smell of colitas, there she stood in the doorway, then she lit up a candle, there were voices down the corridor, The pink champagne on ice, Mirrors on the ceiling, surreal, abstract, minimal --w 1920 --h 1080 --hd - @neo” — it’s so interesting to see the story elements and multi-sensory aspects (“warm smell of colitas”)

• Another fragment for if I write about this: Stilgoe imploring all of us to learn to draw. I’ve always wanted to. This is that but also not that.

• Interesting and (perhaps irrationally?) disappointing to me how many of the images in the Midjourney channels are “genre”—like people just want to be knights or vampires or mermaids. Genre is a lot of life, it’s nothing to be disappointed about. I recently rediscovered what was either a Neil Gaiman or Neal Stephenson remark about genre and the “beats” people look for in a story. I think about that too when my almost-four-year-old son is playing Crayola Create & Play on the iPad Mini, and there’s an egg he can “crack” that holds a surprise inside. Few beats more powerful than an egg about to crack. Also makes me think of the Rachel Maddow piece I read today—her new production company is called “Surprise Inside.”

Okay, that was twenty minutes! I think next up will be making my own images. But first, a five-minute break.

10:06am

Time to sign up to use Midjourney on my own! I grabbed some tortilla chips from the pantry.

I feel it would probably be more courageous to just use it in the public channels, but I relate to community energy in a complicated enough way that I want to isolate out experiencing the tool from participating in the community.

What is it with people putting “unreal engine” in their Midjourney prompts?

As I go to buy a subscription, pretty interesting to see these details:

The beginning! I like that the command is “/imagine.”

Oh, funny—I asked for a scene with a coaster in it and it converted the first part of the prompt to being on a coaster.

Wow, this is not what I expected but I kinda like it.

10:33am

I’m about to embark on the last twenty-minute sprint I’d planned for exploration and playing around. While it’s not yet “the afternoon,” I want to take some of this time while our nanny is out with the kids to be in the kitchen and make myself some lunch from the cookbook Snacks for Dinner (recommended by Robin Sloan in the most recent Fat Gold newsletter, actually!) I’ve never been big on cooking but I’ve recently realized that making something in the kitchen helps to reduce my resentment for cleaning up the kitchen, so I try to pair the two activities now. I think I’m going to make some marinated beans. Step one: boil water to rehydrate the sundried tomatoes.

This afternoon, my concept is to break the seal on doing some writing. What writing? I don’t really know! Or I’m afraid to say, for fear of jinxing it. I would like to come out of this week with a solid many-thousand-word draft, keyed to my true thoughts on some things I’ve been thinking about relating to creativity, generativity, time management, and a few other topics, too.

I think I have to try Sudowrite before I really know what there is to say. It’s made by an old friend and I’ve somehow never even opened it, which really makes me feel like I can’t possibly be living my values.

Interesting that their headline is about busting writer’s block. I’ve thought for a while that there’s some serious delicacy required in describing AI-assisted creativity, but the main applications will be the less creative ones. What percentage of Photoshop usage is removing blemishes from photographs? And should I be demoralized by that?

(Brian Eno writes quite a bit about Photoshop and Kid Pix in his 1995 diary. How I loved Kid Pix! I was eight years old in 1995.)

(I wonder who made Kid Pix? Does there exist an oral history of it? A bookmark for the future.)

One thing feels clear: I really must reach out to Robin Sloan. Maybe writing to him can be my capstone for today.

Soft gradients are everywhere now. Where did it begin? With Instagram’s logo refresh, or before? Who has written about gradients in the 2020s?

Could Midjourney make some beautiful gradients? What makes a gradient beautiful?

Something I’ve noticed about my preference for horizontal tools: I hate to be boxed in, and horizontal tools make me think there’s always a surprise waiting around the corner. (Whether or not there is.) Right back to surprise eggs. I should probably write about the LEGO set insight, too. (Tool builders always want to say they’re creating a box of LEGOs, but even LEGO mainly sells LEGOs in sets with very specific instructions for how to put the pieces together.)

All the sensory details—this is interesting. It makes me think of genre again. I bet this is going to be an amazing tool for writing fan fiction.

One of my recent preoccupations: cookbooks and how they’re the ultimate cozy material. No wonder people like to read them for fun—there’s no tension, only the fantasy of delicious food and loved ones joining together.

Could Sudowrite write recipes?

How do recipes relate to the LEGO set idea?

12:01pm

Well, I’m back from my hour-long early “lunch” break. I did a few things:

• Made the marinated bean dish from Snacks for Dinner. It cooled down and I took the first bite just a moment ago. Delicious! I thought it would be not that much more exciting than my regular no-cook vinegary bean salad, but it’s way different. I’m so excited to have something new and so tasty to try. The crushed red pepper flakes really make it.

• Started a load of dishes, including taking the washi tape labels off of the last of the ceramics that I’d used for stashing marble run pieces in over the weekend when I built an elaborate plastic marble run (from a kit) for the kids. Used nearly every bowl and plate in the house.

• Took a load of compost from the freezer out to the garage, so that at least we’re not backed up on compostable stuff. We are very backed up on bags and boxes that need to be taken out for recycling, but at least those don’t smell bad or take up precious limited space in the freezer.

• Found myself with twenty minutes remaining before the next clock hour began, so spent ten of them reading the rare paper book: Callas Kissed Me…Lenny Too!, by an art critic. Always makes me feel delirious with purpose: I’m living the life of my dreams.

New idea that hit me as I came back from my lunch break: can Midjourney or DALL-E generate interfaces? “Futuristic productivity software”? Let’s try.

Oh, and in what must be a coincidence but doesn’t feel like one: I JUST got DALL-E access granted to me a few minutes ago.

Oh, here are the gradient results from earlier:

Interesting to see the model respond to value judgments like “beautiful.”

I don’t think Midjourney uses transfer (I think that’s what it’s called when one prompt informs the next), but that sure seems gradient-inspired. Perhaps reinforces my claim that gradients are the dominant design touch of this generation of software.

Now I want to test some of these same prompts on DALL-E.

It’s very interesting to me that DALL-E seems to “get” the lingua franca much better than Midjourney. But the Midjourney ones seem a lot artier by default. I wonder if there’s anything in the Midjourney model that prioritizes that? Midjourney stuff does seem to all have a similar aesthetic by default.

Wow, I find these quite beautiful.

“Elaborate marble run in the style of kidpix,” from DALL-E:

Interesting that the DALL-E interface includes a button that says “Surprise me.” “Surprise” is definitely a key word in AI-powered (or -assisted) creativity. What are the verbs we use for talking about AI?

“Blueprint for a marble run design that has never been seen before” (DALL-E)

Wow, “blueprint” seems like a good keyword—I love these.

This one in particular is very beautiful to me.

I think it would be fun to paint or draw by projecting these generated designs onto paper somehow and tracing them from scratch. Then, you get the weird imagination embedded in the original but also the texture of the human hand.

Interesting that I’m drawn to DALL-E more already—maybe because it seems like there’s more variation in the style of the output without me having to describe much.

1:02pm

Well, that last stint marked my sixth twenty-minute sprint of the day—gosh it feels like a long day already! I’m having a blast but I would also love to really sink into something. Maybe some reading.

I did find one article with the Midjourney founder, and now I’m deep into reading a more detailed interview. In it, he shares that “we have a default style and look, and it’s artistic and beautiful, and it’s hard to push [the model] away from that, meaning you can’t really force it to make a deepfake right now.” That’s interesting on a few levels. “And personally, I don’t think the world needs more deepfakes, but it does need more beautiful things, so we’re focused toward making everything beautiful and artistic looking.”

Every image costs money. Every image is generated on a $20,000 server, and we have to rent those servers by the minute. I think there’s never been a service for consumers where they’re using thousands of trillions of operations in the course of 15 minutes without thinking about it. Probably by a factor of 10, I’d say it’s more compute than anything your average consumer has touched. It’s actually kind of crazy.

I think the style would be a bit whimsical and abstract and weird, and it tends to blend things in ways you might not ask, in ways that are surprising and beautiful. It tends to use a lot of blues and oranges. It has some favorite colors and some favorite faces. If you give it a really vague instruction, it has to go to its favorites.

And then there’s a whole stretch at the end that’s a bit grand but is also fresh to me:

And one day, I did that: I made 40,000 pictures in a few minutes, and all of a sudden, I had this huge breadth of nature in front of me — all these different creatures and environments — and it took me four hours just to get through it all, and in that process, I felt like I was drowning. I felt like I was a tiny child, looking into the deep end of a pool, like, knowing I couldn’t swim and having this sense of the depth of the water. And all of sudden, [Midjourney] didn’t feel like an engine but like a torrent of water. And it took me a few weeks to process, and I thought about it and thought about it, and I realized that — you know what? — this is actually water.

Right now, people totally misunderstand what AI is. They see it as a tiger. A tiger is dangerous. It might eat me. It’s an adversary. And there’s danger in water, too — you can drown in it — but the danger of a flowing river of water is very different to the danger of a tiger. Water is dangerous, yes, but you can also swim in it, you can make boats, you can dam it and make electricity. Water is dangerous, but it’s also a driver of civilization, and we are better off as humans who know how to live with and work with water. It’s an opportunity. It has no will, it has no spite, and yes, you can drown in it, but that doesn’t mean we should ban water. And when you discover a new source of water, it’s a really good thing.

It’s interesting to me that so many (“so many”—it’s not that many, but a lot I know about) exited founders end up going into generative AI. “I founded Leap Motion and ran that for 12 years, [but] eventually, I was looking for a different environment instead of a big venture-backed company, and I left to start Midjourney. Right now, it’s pretty small — we’re like 10 people, we have no investors, and we’re not really financially motivated. We’re not under pressure to sell something or be a public company. It’s just about having a home for the next 10 years to work on cool projects that matter —hopefully not just to me but to the world — and to have fun.” And then Sudowrite is built by two serial founders coming together. And Spellbrush is by one of Benchling’s founders.

1:31pm

What feels obvious to me right now:

• Image generation and genre are wound together. The genre leanings of image generation will lead to underestimation because genre is already distinguished from “fine art.” To do: read more about genre.

• I reacted much, much more positively to the model output once it was in a nameable style that I personally like—blueprint and/or 1930s print ephemera. I felt a glimmer of that, too, when I saw someone’s posts last week about “Wes Anderson fashion show” pics generated with Midjourney. I’m aesthetically drawn to Wes Anderson’s work, so was more curious and optimistic about that output.

• I think everyone should have to generate images in a style they personally like before giving up on image generation. Looking at other people’s images is like looking into other people’s subconsciousnesses—a little unseemly.

• I will certainly use one of these models the next time I need an illustration for a blog post. They’re much better than anything I could come up with quickly. And isn’t that interesting, that even though I have a strong streak of skepticism about this, I instantly accept that it could be better than nothing for a pretty important-to-me area of life? I hesitate to call it “art,” but it’s better as an illustration than random photographs off my camera roll.

• I would love to see Sudowrite add illustration as a feature.

• Light source control and text-on-image generation seem like two clear shortcomings of current models.

I actually really like that one in the upper left. I’ll try upscaling it.

Wow, I wonder what I ran afoul of here?

Here’s what I get from Midjourney:

I find this one quite poetic—there is no detailed plan.

Something that reminds me of, from all the articles about Midjourney: emergent pattern of including the prompt that generated an image in its caption.

2:07pm

My brain is starting to feel somewhat fried. I guess that makes sense; I’ve done eight twenty-minute stints and none of them has been particularly passive.

Remembering Robin Sloan’s exercise of inviting his readers to tap through AI-generated output and flag the most interesting ones. Training for interestingness.

I had a goal of doing twelve twenty-minute stints today, but I think I may need to take some sort of break to integrate it all. Before I do, I should probably load up the next batch of reading, since reading is easier for me to do in a background thread.

I want to find the best possible book on genre. Maybe it will be a book on fan fiction.

Are cookbooks coziness fan fiction?

Aha, by Stephanie Burt! I know her! (But just from the internet.) https://www.newyorker.com/books/page-turner/the-promise-and-potential-of-fan-fiction

Interesting that there’s a genre (from Tumblr, no less!) called “imagine”: https://fanlore.org/wiki/Imagine_(genre)

What’s the current business status of Wattpad? Looks like it got acquired by a South Korean conglomerate in January 2021: https://techcrunch.com/2021/01/19/wattpad-the-storytelling-platform-is-selling-to-south-koreas-naver-for-600-million/

From the New Yorker piece by Stephanie Burt:

Fanfic can, of course, pay homage to source texts, and let us imagine more life in their worlds; it can be like going back to a restaurant you loved, or like learning to cook that restaurant’s food.

Cookbooks as fan fic.

2:39pm

My brain is really starting to be fried now—I can feel it slowing down. But I want to make one last push on filling it with material before I take a longer break. It would probably be wise to take a walk, although part of me wonders if I shouldn’t just save that for this evening with the kids. (It’s beautiful out, so I could possibly take them out for an evening playground outing.)

One thing I feel strongly today: I am sort of aghast at the fact that I missed out on material from some of the people I admire most. I mean, I’m still in disbelief that I missed out on the Multiverse development diary. And just today I realized that Lina (the cofounder of Infinity, my one big investment last year) has a blog she’s been writing for years. https://linacolucci.com/blog/ I really have to make sure to be a completionist about the collected works of people I see as great thinkers.

2:59pm

Time is going more quickly now that I’m finding things to read. I spent much of the last twenty minutes down the rabbit hole of the Rec Center newsletter, linked in the Stephanie Burt piece: https://buttondown.email/the-rec-center (But the link was broken—it pointed to a defunct version on TinyLetter—but I was able to find the original.) The latest edition of Rec Center also pointed me toward this essay about the final season of Buffy and reworkings of it in fanfic: https://www.thesmartset.com/textomancy/

But I should probably go back to the original prompt of books about genre.

I’m struggling to find a canonical book about genre fiction. I went down the path of trying to find something by Henry Jenkins, but none of his books seems quite right. I did dig up the Neil Gaiman (it was Gaiman, not Stephenson) interview I referenced earlier and that’s stuck with me for a long time: https://www.newstatesman.com/culture/2015/06/neil-gaiman-kazuo-ishiguro-interview-literature-genre-machines-can-toil-they-can-t-imagine

3:33pm

Okay, I just re-read the whole Gaiman & Ishiguro piece. It’s so satisfying to read again because I’ve had snippets from it popping up in my Readwise daily emails for months / years, but I’d forgotten the whole swoosh of the piece.

Some loose ends before my brain takes a break:

• What does it mean to sit down and be creative? “Deep work” / “time blocks” have a role to play, but it’s not only that.

• Fan fiction also includes an aspect of approbation. This also reminds me of the poems I wrote for publication on alloy.com once upon a time.

• How do creativity and capitalism relate to each other?

• One thing AI is really good for: generating many plausible variants to speed up the act of choosing. Variation and selection; The Beginning of Infinity.

• I find it somewhat disappointing to think about how much of life is genre, but what if I saw it as a good thing? Genre is cozy, and coziness is a high value of mine. Are the mom blogs I read every morning not a genre?

• This is going to be vague, but: I have a bittersweet feeling as I return to all of this source material. It’s difficult to relate to anything anymore as a pure fan, because I’ve been too close to the engine in most every case. I read something, will live in the memory of revering the writer, then snap back to the present truth of how I met that person once (or twice, or many times) and it was all quite complicated in the end. I guess maybe I can find a way to love what’s complicated, too.

Okay, I think time to wrap up for now. I don’t know exactly what I’ll do next, but I will say that I’m truly amazed by how long and lovely today has felt. I can’t believe that it’s only mid-afternoon and there are still several hours before I’ll be back with the kids. I suppose I would feel listless at some point if every day were like today, but right now that’s hard to really believe. Right now, I think I’d just love it.

4:09pm

Brain completely fried and I’m not going to try to have any new ideas, but if some arrive, I’ll note them down.

(Decided to start watching Sandman, on Netflix, in honor of the Neil Gaiman piece.)

• Getting an image back from Midjourney or a paragraph back in Sudowrite feels almost like Yuletide in the fan fic world (which I just read about today, where writers create gifts for each other written, in a way, to spec)—except with instant gratification, endless micromanagement of the AI, and no sense of human effort making it feel more like a gift.

5:14pm

Well, this is both embarrassing and somehow comforting and full-circle: I just went back to Robin’s piece I was loosely referencing earlier today (the one about working doggedly with a single tool / model over time) and realized that in fact the whole thing is called NOTES ON A GENRE (on a genre!) and even formatted as bullets, which much of today’s log is as well. I guess that had been percolating in the back of my mind—my own mental model, maybe—and spat itself out as something nearly-new. What’s funny is that I was aware of the influence! But I was not consciously aware of all its surface area.

I guess now I have more to cite!

August 10, 2022

9:54am

It’s the next morning, and yesterday’s deep dive now feels like a dream. But there are some threads I can tug on, and one of the gifts of a day like yesterday is that now every thirty-minute gap on my calendar feels like it contains the potential for a FULL TWENTY-MINUTE SPRINT that I’d know what to do with since I have so much context loaded into my mind—what abundance!

I’m wishing (intending? tracking toward?) that I’ll sit down and “really write” something tomorrow. It’s not totally clear what, but I think I have enough themes swirling—Kid Pix, fan fic, Brian Eno, different LLMs and their different default aesthetics. There’s also a theme of “giftedness” that I want to find a way to work back in—I have very, very dim memories of studying similar things in a philosophy course in college. (And maybe the thing to do—and this makes my heart skip a bit of a beat–is to reach back out to my philosophy professor, who I haven’t been in touch with for years but who I did write a very thoughtful email to circa 2009 that’s burned into my memory, so reactivating that thread would be okay.) I wonder if he’s even still at Harvard? Ah yes, he is! https://philosophy.fas.harvard.edu/people/richard-moran And now I want to read all of his work. He studies AESTHETICS, too! I wonder if he has an opinion on generative AI? It could be a fascinating correspondence.

My other source on “giftedness” is Peter Singer, who I’m not otherwise a student of but who I did attend one book talk by during college, and he was talking about the “giftedness” of life and how it’s an important counterweight to designer babies. And then all of this also connects somehow to approbation, which I also studied in Richard Moran’s philosophy class. One of my great sadnesses from school is that I didn’t go back to pick up my handwritten exam essay in a little blue book that was all about approbation—it had resonance in my life at the time and was related to something emotionally intense that was going on, so I remember hunching over and writing that essay with the full fire of personal context but all of the expression was academic. I remember being especially proud of it, but now it is lost. But perhaps somewhere on one hard drive or cloud storage destination I have my other essays from that course.

Probably there is space here, too, for The Gift, by Lewis Hyde—a book I never got into but that several people I really respect always loved.

Oh, and Rob Walker also seems like a relevant thinker here! I don’t think he’s written anything on generative AI, but he’s in general one of my favorite thinkers and it would be interesting to correspond with him on this. I think (though I’ll have to search to really know) that Rob once mentioned The Gift to me.

Well, now I obviously have to read The Gift. I already bought it on Kindle once upon a time, so it shouldn’t be too hard to pull up—in fact, I have it loading in another tab. The key will be to find a way to find a way to use it without making “reading the whole thing” an impediment to “really writing.”

In other threads…

One of my loose ends from yesterday was to see if there existed any sort of oral history of Kid Pix. I searched this morning while I was procrastinating a bit while going in to the office and lo! Yes! The creator of Kid Pix wrote a whole post about it! http://red-green-blue.com/kid-pix-the-early-years I also have some hope that I could actually reach out to Craig, but I’ll first need to know about what. In general, if I were to have a hot take in this moment, it would be “everyone talks about HyperCard, but who’s talking about Kid Pix?”

In 1984 Apple released the Macintosh. Up to this point I saw no reason to stop using my Atari (except for the fact that hardly anyone else used them and there was practically no audience for the software I was writing). For its day, it had great graphics and sound, and I knew a lot about programming it. Other computers had more serious reputations but they didn't seem as much fun to program. The Macintosh was different. It was totally graphical so the visual possibilities were much greater. Most of all, the Mac seemed to have people with a consistent and enlightened vision behind it. The user interface was intelligent and beautiful. I got a Mac and began teaching myself to program on it. Originally I programmed in Rascal, a language developed at Reed College which was sort of a cross between C and Pascal. Later I switched to Pascal.

A programming language called Rascal!!! I love it so much.

The program should, in some way, expand the concept of what computer paint programs are, as well as what mark making can be.

I feel a little internal shiver at that—what mark making can be. Feels like there’s either a body of work on “mark making” just waiting for me, or I could be a major contributor to it. And I think “mark making” has a lot to do with generative AI.

In addition to "traditional" computer paint program tools like the brush, pencil, eraser, circle and rectangle, my program would include tools that would be surprising and visually unusual. Most computer paint programs were designed in a way that made them seem like second class citizens, always attempting to mimic traditional art making tools, but never doing a particularly good job of it. In fact, the computer can make marks in a variety of interesting ways that traditional tools can't. It can also do a good job of creating comic tools. This is where the "wacky brushes" came from.

Wacky brushes!!!

The Undo Guy—I remember this guy.

Children love stickers and rubber stamps. The program should allow for some pre-made iconic art. The stamps in the Original Kid Pix were taken from the Cairo font from Apple Computer designed by Susan Kare. Apple had stopped distributing Cairo with their computers and I thought the pictures were too good not to be seen.

Ahhhh I wondered where Susan Kare would fit into this. As a sidenote, learning in my early twenties that Susan Kare was available for art commissions was a singularly disruptive moment in my life. First, it was very exciting: computer history heroes are all around us and we can talk with them—computer history is awfully recent. Second, though, it was the first instance I can remember of realizing that even heroes need to continue making a living, unless they are independently wealthy or participate so strongly in equity upside that they become “post-economic.” And in fact, it’s somewhat hard to figure out what to do once you’re medium-post-economic, which is a state quite a few people in Silicon Valley end up in and I know a lot of them now. You don’t necessarily need to work for pay in any given moment, but you probably need to at some point—so at what point? And is there a way to work intensely but in compact pockets so that you can still, say, look after your health and exercise and meditate daily in a way you never did when you were in the thick of your signature win? This also turns out to be a challenge for even very famous athletes. (Although they at least have the “working out” bit down.)

I could have run the Kid Pix name together to become KidPix, as was the style then for program names. I decided against it because I was afraid it was just a naming fad, although today if you do a search on the web the name appears as much one way as it does the other.

So interesting that that came up even at the time—I wondered about that several times when I was typing the name, since I’m sure I’ve seen it both ways, and now I know why.

Oh wow, this is so delightful to me:

It was the style at that time to add the word "Professional" to new versions of software. To be humorous, I called the new version "Kid Pix Professional."

Wow, his son Ben looks so much like Cooper. It’s actually eerie.

I ended up selling about 100 copies of Kid Pix Professional and think that if vintage software is ever collected, it should make one of the really rare finds.

He eventually links up with Broderbund:

Most stressful was dealing with bugs. Broderbund had a large department of skilled people whose job it was to find things wrong with software. And they did. I had thought Kid Pix Professional was quite bug free: after all, I had been selling it. My first bug report was 30 pages long! And each new feature that was added brought new bugs, which had to be fixed. Even the bug fixes often created new bugs. In the past I’d only had myself to please with my programming; now I was responsible for fixing all this myself. In the end each one got stomped out. This was also a stressful time because I was teaching full-time at the University of Oregon in a department that required many hours of teaching and committee work.

Early Apple keynote vibes:

…in January, 1991 John Sculley featured Kid Pix in his keynote speech at MacWorld. In the following issue of Macweek a headline read, "Kid Pix Vrooms past Sculley at Expo."

Many many loose ends to chase here but I’m way over my twenty-minute block. I’ll leave this one link that I found while searching for Leslie Wilson, the Broderbund product manager who adopted Kid Pix: https://www.latimes.com/archives/la-xpm-1999-jun-28-fi-51399-story.html

There are also many videos on Craig’s About page, some of which are Kid Pix-related interviews: http://red-green-blue.com/about

Okay okay, one more thing. Let’s break the seal on “mark making.” Lots of results on the first page of Google Search, here’s one: https://www.ubelarts.com/blog/mark-making-inspired-by-the-masters-to-find-artistic-voice/ Seems like it’s an art term. But a quick search for “mark making DALL-E” shows that I don’t think it’s been used to describe at least that LLM. Opportunity to connect dots!

12:14pm

I only have maybe ten minutes, but this journal is starting to become more magnetic for me, which is what I hoped for—so I want to keep up the momentum. I WANT to be here in between things, and everything else feels like a bit of a distraction.

Maybe in this ten-minute block I can discover all the references to Lewis Hyde in my long-ago inbox.

Wow, I did not quite find what I was looking for, but I did find a long email I wrote Rob Walker (referenced above!) in response to the PDF he sent me of his book The Art of Noticing. I think none of it’s secret, so I will include it here because it‘s remarkable HOW MANY OF THE SAME THEMES also show up there, despite coming from January 2019:

The topic of noticing brings to mind my favorite college professor, John Stilgoe. He’s the sole professor of “environmental studies” in the art department at Harvard, which for him means the study of the built environment. He’s a classic character; he wears bow ties, is stern, and keeps the university on its toes by insisting on continuing to use a slide projector when the rest of the university has transitioned to digital. (He also insists that university communication be sent to him by snail mail, so that the university is forced to maintain a physical record of its correspondence.) Here’s an interview he gave to 60 Minutes (apparently facilitated by a former student…he has quite the alumni network), and here’s a video he did as Harvard propaganda. (Note I’m just link-grabbing, not confirming those videos have the right content by watching them.) I remember he told us that the 60 Minutes piece includes him telling the baffled interviewer about the arrow in the FedEx logo, hidden in plain sight. He’s been known to give students the assignment to go into Harvard Yard when it’s dry out and map out where they believe the puddles will form. He wrote a book about the power of bicycling to change your perspective, Outside Lies Magic, and a book called The Shallow Water Dictionary, which draws attention to the fading natural world around us by sharing words that are rarely used anymore. He told us that the dictionary had done quite well as an impulse purchase next to museum checkouts. I took four classes with him in college, and they’re about all I remember of college at this point. If you haven’t yet heard of him, he’s worth seeking out.

• I loved the patterns in the book of adapting art assignments by making them easier, and then reverse-engineering art that already exists. So smart.

• I had a fun time imagining the process of you writing this book. How did the book proposal come about, I wondered? Was it conceived as a gift book? Who was the imagined audience? What was the imagined blockbuster success case? Will you give talks to art classes or corporations on this, or is it not designed for the speaking circuit? Will you assemble an INSPIRATIONAL KEYNOTE? How did you collect anecdotes? (Shamelessly: what software did you use?) Who were your best sources? Was this your small talk topic for a few years? How did you assemble your reading list? What were the hard parts that most people wouldn’t even think of? I also noted, with some pleasure, that this book drew heavily on The Hermeneutic Circle folks. And only we will ever know!

• A pleasing ego note: I spent a few minutes imagining how I could have led my life to become one of the few who won MULTIPLE mentions in the book. What would it mean to be a champion noticer?

• Along those lines, too, I noticed myself craving a storyline where this book was one of personal redemption for you. How, I wondered, were you changed by the process? Was the impulse for the topic commercial (it will sell), curious-yet-detached, or self-healing? I’ve been deep in the genre of self-improvement-alogues lately (such as Happier at Home and Bringing Up Bébé), and part of me wanted this to be explicitly about your quest for greater presence.

• This book, and topic, would have been great in the golden age of Tumblr. I’ve been thinking lately that Instagram is the new Tumblr.

• Along those lines, “paying attention to what I pay attention to” has me noticing how preoccupied I am by indie ecommerce driven by Instagram presences. Elizabeth Suzann and Tuesday Bassen are my best examples. It occasionally makes me think maybe I should go work at Instagram one day, but I know from working on other consumer apps that much (so much) of the company’s time and energy ends up being spent holding back the tides of internet awfulness. I’m sure working at Instagram is only minimally about pleasant indie ecommerce, and mainly about big brands, celebrities, and abuse.

• I found myself longing to know what ended up on the cutting room floor. What 131 other strategies didn’t make the cut?

• A recurring pattern in the book: a quotidian project + persistence can = minimum viable fame. Many of your examples came from people who did just that. The #100dayproject is a practice that systematizes something like that. Enough persistence, and you’ll get noticed.

• Speaking as someone who went through executive coach training (and loved it): a good interviewer is a lot like a good coach. The questions in the StoryCorp section would all make great coaching questions.

• I gotta read Man’s Search for Meaning.

• Apropos of the last essay: “The Importance of What We Care About,” by Harry Frankfurt (author of On Bullshit) is one of my all-time favorite pieces

• The section on wasting time on the internet brought to mind this great Radiolab episode about “where story ideas come from,” wherein one contributor shares two amazing strategies: 1) Google Alerts set for phrases that capture little narratives (“the human equivalent of”) 2) really, really, really deep Wikipedia rabbitholing. (Here’s a Transom piece that shares many of the same strategies.)

Wow this is bizarro, I have an email-to-self from April 2013 that goes like so:

It's a lot of responsibility to have other people's hopes and dreams tied up in your own. But also, what a gift! 

READ Lewis Hyde's The Gift, pull out something related to phonebook pages

$1 backers true fans, or curious onlookers? You can MAKE then true fans by giving them lots of hooks to grab onto. Make the price of expressing curiosity low, and the rewards amazingly high.

Dark Sky - persevering through early rough reviews

Type Rider - leveraging a small community, backers as collaborators

A Thousand Natural Shocks

Awkward Black Girl? [ed note: this was a comedy series by Issa Rae before Insecure, I think I was thinking about it because Jenna Wortham was working on an article about Issa Rae]

Celebrating Moments (receipt around the neck, etc.)

Bechtdel / Bechst (chiropractor / orthodontist?)

I sort of love that I implore myself to “READ Lewis Hyde’s The Gift,” but what does “pull out something related to phonebook pages” MEAN?

Ten minutes up, but glad I could make even ten minutes a worthy rabbit hole.

12:31pm

Waiting for a call to start, but a quick observation from opening up The Gift on my phone: it’s quite possibly part of an answer to “how does creativity relate to capitalism.”

1:05pm

One call complete, half-hour gap begun, twenty-minute timer started. It is so delightful to see time as richer and denser in this way.

I could go back to The Gift but since I’m at my computer I’m interested in doing something more computer-y. Maybe I can generate a few more images.

To “get the wheels turning” it often (“often” as in today and yesterday, ha) helps to start with a scene that I lived through or noticed, just to appreciate seeing my own memories refracted through a model. Of those, the top right one seems very strange and surrealistic—who is sitting where? And how interesting that the “mom” appears to be “holding” something like an infant.

It’s also interesting to see it in Midjourney because the colors that happened to be true to my own life (auburn, aquamarine) also happen to line up with the color scheme Midjourney likes—blues and oranges.

Wow, that one is sort of disturbing when scaled up.

But would it look very out of place at a museum? It would not. I would try to find meaning in why the pile in the lady’s lap could be a child, a book, a blanket, or all three at once.

This output from DALL-E looks quite a bit more like I was expecting:

Wow, I’m pretty impressed that DALL-E got “old print ephemera” right:

Okay, this is kind of awesome:

Very “Midjourney” in aesthetic:

Wow this is kind of grim:

Now we’re talking:

I’m loosely playing now with “what do I want the illustration for this essay to be?”, but also as a way to see what I want the piece to be about. (If I hit on a really good image, it would make me more excited to make those keywords the fulcrum of the piece.)

Interesting too that while I could get each model to run itself many more times to generate more output to approximate more of what I’m looking for, my instinct is instead to switch models. “No, that won’t do! Back to the drawing board!” (Could “drawing board” be a useful keyword here?)

I still want to see what would happen if I fed Sudowrite text into these models to get “illustrations.”

Still feeling sure that if I printed some of these out and traced over them in mixed media, I would feel happy about the output, like it was “really art” that I was a real collaborator on—through effort and mark making.

5:29pm

A few thoughts that arrived when I relaxed my mind:

• So far, I see at least four ways to look at AI-generated images: as genre, as illustration, as warm-up, and as lucid dreaming.

• What feels “not art” about AI-generated images is not that they’re not good, but that they’re not gifts, in the Lewis Hyde sense.

• The question of “art” is a distraction from what’s really going on here; there are plenty of other uses for images. (Part of why I think “illustration” is a powerful lens.)

• There is a paradox in AI-generated images: they are both all-surprise and devoid of surprise. (“Surprise me,” but only with shades of everything that’s ever been seen.) This is related to Robin’s points in NOTES ON GENRE.

• Part of what we like and dislike about AI-generated imagery is the opportunity to micromanage.

• “To spec” and “spec work” could be an interesting lens here, too.

• I am getting the sense that this line of thought combines everything I’ve ever thought about, which may be evidence that LLMs are something of a Rorschach test. To explore: the history of Rorschach tests.

6:07pm

• As emoji

• As toy

• As alethiometer

• As stickers

• As wacky brushes

August 11, 2022

8:45am

A stray thought: prompts as captions, captions as prompts. Looking into the theory of photography and the art of captions.

Multimodal artists are also a key to understanding LLMs.

9:27am

I’m in a bit of a pickle: I’ve designated today a “writing day” without perfectly specifying what that means. So I’m experiencing all the butterflies of gearing up to “really write” without the direction of knowing what the pressure’s all about.

I guess I can first share an update of all that’s transpired:

• I’ve been reading more of The Gift, by Lewis Hyde, in the Kindle app on my phone. The introduction felt acutely relevant to the topic at hand (which is…?), the first proper chapter less so, and I feel the options are to keep going or just to accept the material of the first chapter. This would be an easier decision to make if I didn’t have other relevant books to parse, but I do because…

• My search for a book about Rorschach tests paid off! It turns out there’s a quite good and recent book called The Inkblots that even includes the tempting subtitle Hermann Rorschach, His Iconic Test, and the Power of Seeing. The Power of Seeing! What does it mean? Could it hold the key?

• The line of thought about captions (above) makes me think I should probably read a whole book about photography but also that I’m probably procrastiresearching. But can it be procrastiresearching if it’s only the second full day? Surely I’d benefit from being in the thick of research a bit longer? But it feels so indulgent. But it’s also a cheap luxury relative to a lot of things in life. A day spent reading is pure joy and the cost is purely measured in opportunity cost. But everything has opportunity costs, it’s just that most things also have monetary costs.

• I would like to let myself linger in research a bit longer while also pressuring myself to at least break the seal on writing, so that it doesn’t loom too large in my imagination. So I’ve set myself a goal of 1,000 words of shitty first draft today. This document holds 8500 words already, so surely 1,000 should be possible.

• I decided to repeat Tuesday’s experience of selecting a recipe from Snacks for Dinner to make myself for lunch—this time, an eggplant tahini chickpea dip with Mary’s Gone Crackers. I’m sorry to say that the grocery order came to about $70, which seems like an awful lot for eggplant tahini chickpea dip. Arguably I will only use a small amount of several of the ingredients, but then in order for them not to be a waste I must come up with other uses. As Lewis Hyde says, “the gift must stay in motion”—especially true with consumables (which make especially good gifts).

• I only have one phone call today, but it’s coming up quickly, thereby delaying the “start” of the day some. Still, I’ll return to the observation that I’m quite confused about “what my goals even are.” I want to write a grand unifying theory of AI art that also happens to be a semi-lyrical personal essay. But to what end? Proving to myself that I can still do it, and that I’m different? What an odd way to spend time. But if I have total time freedom and I still don’t do that kind of thing, who even am I? Certainly I wasn’t born to be on Zoom all day. (But let’s not forget that meeting people in the first place leads to knowing a few people deeply, which is a big source of meaning in my life. I can’t know people I never meet.)

• Now I’m really procrastinating, but the idea of being “born” for something reminds me of one of my son Cooper’s odd / insightful phrases: he talks about how I built him and he built me, which is in a way true because he made me a mom, which is now a big part of my identity. We’ll just be lounging together, cuddling, and he’ll suddenly talk about how he built me. The mutuality of it really gets me.

Three minutes left in this twenty-minute block. I did a great job burning time!

From The Inkblots:

This double biography of the doctor and his inkblots begins in Switzerland but reaches around the globe, and down into the core of what we are doing every time we look and see.

I am a total sucker for such grand proclamations. But what ARE we doing every time we look and see?

Okay, time’s up.

12:20pm

Several things have transpired:

A phone call

• A kitchen interlude 11-11:30am—completely delightful to poke away at dishes WHILE composing a dish in a quiet house. Creating / cleaning as a pair. Broiled eggplant for the eggplant dip. Discovered that the toaster oven could broil. (The setting was there all along, but I thought it wasn’t possible because I didn’t look at all the menu options and on the “bake” setting, the oven doesn’t go higher than 450 degrees. But set it to broil and the 500th degree is unlocked! Always read the menu!) (Also our ancient big oven is broken.)

• A return to my computer 11:30am-12:10pm or so, where I discovered an email thread that needed to be hopped on right away.

• 12:10-12:20pm: Go back upstairs to see if the eggplant has cooled in the fridge. Yes! That didn’t take long at all. Make dip. Delicious. I think snacks for dinner / snacks for every meal might just be our family way. I wish I’d ordered some pickles, and then we could make a picnic dinner out of all of it.

• A question: how can I enjoy evenings more? I always seem to run out of energy.

But that’s just regular journaling. Let’s get back to generativity journaling.

Well, one thing I haven’t mentioned about generativity yet is that I’ve sometimes said it’s one of the qualities I value most in people—particularly in the context of product manager interviews, but I think it’s also true of founders. The ability to think on one’s feet, improvise, come up with something plausible on the first try—I mean, what could be more fun? And it’s one of the qualities people have said they value in me: the ability to come up with ideas quickly, so quickly that we can throw a lot of them away and I’m not at all married to any of them. The looseness and play that comes from non-attachment, and non-attachment comes from abundance and speed.

So could AI-generated images hold some of that feeling of non-attachment? They certainly feel like toys. But they don’t carry any of the gift of a quick mind working quickly. They do work quickly, and quick minds made the models—but the quickness is a brute force thing. Would I like AI-generated images better if they came from a smaller model? Probably, yes. Which means I should probably try out mini DALL-E and Craiyon.

(Oh: I’d heard both names, but actually they’re the same thing.)

(Ad-supported.)

Here’s the “model card” on Hugging Face for this model:

https://huggingface.co/dalle-mini/dalle-mini

The model could also be used for downstream use cases, including:

• Research efforts, such as probing and better understanding the limitations and biases of generative models to further improve the state of science

• Development of educational or creative tools

• Generation of artwork and use in design and artistic processes.

• Other uses that are newly discovered by users. This currently includes poetry illustration (give a poem as prompt), fan art (putting a character in various other visual universes), visual puns, fairy tale illustrations (give a fantasy situation as prompt), concept mashups (applying a texture to something completely different), style transfers (portraits in the style of), … We hope you will find your own application!

Interesting that fan art figures prominently here as it did in my own observations of the Midjourney Discord.

Interesting that these are actually more realistic and less stylized than the blueprints I got from the more advanced models.

A podcast with Boris Dayma, one of the creators of DALL-E mini: Boris Dayma — The Story Behind DALL·E mini, the Viral Phenomenon

Well! Following Boris everywhere. Gotta make time to watch / listen to the whole podcast episode. So glad I’m finally down the “right” rabbit holes. (More specific, more aligned with my interests.)

It’s also interesting that the iconography of Craiyon is an emoji crayon. Right back to AI-generated images as emoji. Also, crayons have been a hot topic around our house lately because Cooper went from playing the Crayola Create & Play app on the iPad Mini (it’s probably stylized “mini”) to being very attached to Crayola as a brand (they got him!) to requesting “real Crayola Create & Play crayons,” so I ordered a box of 64 crayons (it actually turned out to be a two-pack) on Amazon and he’s been amazing me with his creations. They all start the same: sunlight, grass, sky. Oh! That makes me think: I should find a book about how children start to draw. Probably there’s something in there for “this piece.” (The piece I still haven’t started or defined.)

Also, following the AI-generated-images-as-emoji line of thought: I think something that would be shockingly rewarding is giving people ways to respond to other people’s creations with AI-generated images. Now that would feel like a gift. The act of choosing from among the hundreds of emoji when reacting to someone else’s work already carries a feeling of gift when received by the creator (some of the power of emoji reactions in Slack). And carefully-chosen GIFs are the same. So would the stronger form not be “I liked what you said so much that I generated a custom image using practically-supercomputers as a response”?

Also need to find a book about generative music or whatever Brian Eno called it. But time is up.

12:48pm

I’m going to start this twenty-minute block with some book searching, since having the right book on hand makes it a lot easier to keep the research going (and the ideas percolating) in my low-energy evenings.

Ooooh look what I found!!! Multimodal Perspectives of Language, Literacy, and Learning in Early Childhood: The Creative and Critical "Art" of Making Meaning

Multimodal is definitely a key word.

(Another Robin callback: “media inventor” as role.)

The volume’s explicit focus on children’s visual texts (“art”) facilitates understanding of multimodal approaches to language, literacy, and learning.

Well, obviously I’m getting it.

(Just heard the front door open; kids are back, 12:52pm.)

Another thing this makes me think about: in reading the paperwork for Cooper’s upcoming preschool start, I found the phrase “we strive to be a media-free environment”—by which they mean not only no screens, but no named characters. No Paw Patrol, no Barbie, no nothing. Their explanation is that seeing those images affects children’s play. And at 826michigan, children could improvise just about any story except one with a mass media character. Essentially: no fan fiction allowed! So here we have a privileging of “organic” narratives where the component parts have “pure” sources. And certainly the language of folk tales would be welcome. From The Gift by Lewis Hyde:

Folk tales are like collective dreams; they are told in the kind of voice we hear at the edge of sleep, mingling the facts of our lives with their images in the psyche.

That phrase “the kind of voice we hear at the edge of sleep” has stuck with me, in large part because “the voice” is, I think, a parent’s voice (in my case, a mother’s voice) telling a lilting story to a child.

I’m not sure how this will relate, but since I’m dredging up associative thinking: I was noticing yesterday that my children (who of course cannot read) can really only select things on a screen according to images. So selecting songs from a Spotify playlist (which I think of as more virtuous than handing two-year-old Bentley a device loaded up with YouTube Kids) is quite a bit harder than choosing the same songs on YouTube, where she has visual cues because every video has a thumbnail, and not every song on Spotify has a distinguishing thumbnail. Meanwhile, I’ve been using the Amazon Kids+ app with Cooper for bedtime stories (satisfies his desire for novelty), and he very often selects the book images from a grid that has a mass media character he recognizes—Spidey, or even Barbie.

Oh, I also had an idea for a game: in the spirit of Apples to Apples, something where there’s a prompt, you feed it into one of these models, and then you have to guess which of the four images your partner would choose as the one they prefer. Very subtle aesthetic preferences! Or not—maybe we’d “all” like the same ones. (This is my hunch about what happened with Midjourney’s model: as it trained on people’s preferences, it converged on really liking the blue/orange color combo, because “people” really like that combo in color theory. I remember reading about that in an article about Mad Max.)

Looking for a book on generative music to complete / further my book search, I found this book: Learning as a Generative Activity. I don’t think I’ll buy it now, but bookmarking it for a possible future.

Okay, here’s a 1996 talk from Brian Eno on generative music. A fair place to start.

Ummmm

The following talk was given by Brian Eno at the Imagination Conference in San Francisco, June 8, 1996. Billed as a progressive interactive event featuring original multimedia presentations the Imagination conference featured musician and artist Brian Eno, movie producer and director Spike Lee, and performance artist and musician Laurie Anderson.

The Imagination conference?

Okay, before I go any further, I probably need to research the etymology of “Imagination.” Obviously it has “image” in it, but I want to understand more. It’s such a potent word but it also feels to me flattened by the Midjourney “/imagine” command. “Imagine” as command—it just feels wrong. Creativity on command: not possible / corrupted? We want creative acts to be freely chosen. This is back to the philosophy / approbation point—Richard Moran’s class had a specific word for it, what was it?? It wasn’t free will, I don’t think it was chosenness. To the thesaurus I go…I don’t think the word was “volition,” but I do love that word. “The faculty or power of using one's will.” Similar to “voluntary.

This reminds me, I thought of another way to look at AI-generated images last night:

• As thesaurus

2:14pm

Wow, today has been much more chopped-up than I expected. “Things keep coming up,” and I find out about them in my “breaks” between twenty-minute stints. Probably on some level I’m avoiding writing. But let’s work backward: I think in order to write a thousand words of bad first draft today, I need to dedicate at least four twenty-minute bursts to it. That will take probably at least three hours if I assume one more thing will come up in the meantime (that is actually my job, and not a distraction). Which means I need to start…already, sob! And when will I ever do email?? Before 5:45pm was the hope, but I feel like I need more time than that.

Time to tamp down the angst. Let me just go step by step.

Back to “imagine” etymology: https://en.wiktionary.org/wiki/imagine

Something interesting is that, in true wikiverse style, the fandom inflection of “imagine” is on the dictionary page.

(fandom slang) A short fanfic or prompt placing a reader insert in a novel scenario with a character or celebrity.

And back to the Brian Eno piece on generative music:

So I've always wanted to set things in motion that would produce far more than I had predicted.

This, too:

Now a moire pattern is when you overlay two identical grids with one another. Here's one, here's the other. Now when I overlay them, see what happens, you get a very complicated interaction. You get something that actually you wouldn't have predicted from these two original identical sheets of paper. This is actually a very good analog of the Steve Reich piece in action. Something happens because of one's perception rather than because of anything physically happening to these two sheets of plastic which produce an effect that you simply couldn't have expected or predicted.

Infinite potential / length:

The thing about pieces like this of course is that they are actually of almost infinite length if the numbers involved are complex enough. They simply don't ever re-configure in the same way again. This is music for free in a sense. The considerations that are important, then, become questions of how the system works and most important of all what you feed into the system.

Exploratorium connection!

One of the first places I came to was San Francisco, I lived here for a while. In fact I practically lived in the Exploratorium. (Laughter and applause) I have my Exploratorium. instant moire in my pocket (laughter). If you haven't visited the Exploratorium. in the last month you should go — it's really a good place. If every city had one of those the world would be a much better place.

In the Exploratorium. the thing that absolutely hooked me in the same way as the Steve Reich piece had hooked me was a simple computer demonstration. It was the first thing I'd ever seen on a computer actually, of a game invented by an English mathematician called John Conway. The game was called Life. Modest title for a game.

Wanting to train intuition:

At the Exploratorium, I spent literally weeks playing with this thing. Which just goes to show how idle you can be if you're unemployed. I was so fascinated, I wanted to train my intuition to grasp this. I wanted this to become intuitive to me. I wanted to be able to understand this message that I'd found in the Steve Reich piece, in the Riley piece, in my own work, and now in this. Very, very simple rules, clustering together, can produce very complex and actually rather beautiful results. I wanted to do that because I felt that this was the most important new idea of the time.

Time’s up, but one thought: part of what Brian Eno is responding positively to is the creation of complexity from simple rules. And maybe LLMs could be seen as huge complexity generating simple results.

How can we train our intuition to understand LLMs?

Don’t forget about the career of Prompt Designer.

It’s interesting that “prompt” also has the meaning “fast.”

2:43pm

I have to say, I think it’s a pretty good trick that I’ve convinced myself that limited-release docblogging is “not really writing,” therefore not something I resist in the least, therefore I can get a ton of words on the page. But shh, don’t let myself know!

I need to start “really writing” something soon, but I just had a thought cross my mind, connecting something from Eno’s 1995 diary to that folk tale line from Lewis Hyde. Eno said something about how ambient music was supposed to sort of fade into the background but add texture and reward interest when people did feel like tuning in. There’s some similarity there—both flirting with the edges of sleep.

I was getting a lot out of the 1996 talk, so I suppose I want to finish it, though I also feel nervous that I won’t do all the writing I meant to do. But what if it holds the key?

Re: screensavers:

again they picked up that thread of something that uses a tiny amount of information, a minute amount of your computer's processing power, and produces something that for me is thirty times as beautiful as anything I've seen off a huge clunky CD ROM.

I think LLMs might be the “huge clunky” thing in this case. But just because they’re not elegant doesn’t mean they’re not useful. Elegance / glamour / art / gift.

But this certainly sounds more like LLMs:

I quickly realized that for me this was the future for computers. Computers seen not as ways of crunching huge quantities of data or storing enormous ready-made forests of material, but computers are the way of growing little seeds.

On surprise:

This thing will go on generating like this, and it will stay pretty much the same, but it will never be identical. This suits me fine. I don't want big surprises. *I want a certain level of surprise - I'm too old for big surprises, now.

A company called Sseyo:

…sent me a demo of something they had done. It was a music generating system. I listened to this CD and there were a couple of pieces on it that were clearly in my style. In fact it turned out that they were followers of my music. The interesting thing to me was that the pieces that were in my style were actually very good examples of my style. In fact they were rather better than any I had recently done (laughter). I was rather impressed by this.

This reminds me of part of the Midjourney interview where the CEO talks about how artists want to train the model to better mimic their work so that they can use it more in their own creative processes.

A lot of the famous artists who use the platform, they’re all saying the same thing, and it’s really interesting. They say, “I feel like Midjourney is an art student, and it has its own style, and when you invoke my name to create an image, it’s like asking an art student to make something inspired by my art. And generally, as an artist, I want people to be inspired by the things that I make.”

…

We don’t have a process for that yet, but we’re open to it. So far, I would say it doesn’t have that many artists in it. It’s not that deep of a dataset. And the ones who have made it in have been giving us like “we don’t really feel intimidated by this” answers. Right now, it’s so new; I think it makes sense to play it by ear and be dynamic. So we’re constantly talking to people. And actually, the number one request we get right now from artists is they want it to be better at stealing their styles, so they can use it as part of their art flow even better. And that’s been surprising to me.

Repetition as a bore:

Having started working with this system I am so thrilled by it. I think there are other generative music systems, but I happen to understand this one and I know it's a good one. I'm so thrilled by it that it is very difficult for me to listen to records anymore. Putting on a record and knowing I'm going to hear the same thing I did last time has actually become a little bit irksome. It feels quite Victorian to do that (laughter). I think this has really moved up into a new phase of music.

All about generativity:

Classical music, like classical architecture, like many other classical forms, specifies an entity in advance and then builds it. Generative music doesn't do that, it specifies a set of rules and then lets them make the thing. In the words of Kevin Kelly's great book, generative music is out of control, classical music is under control.

Now, out of control means you don't know quite what it's apt to do. It has its own life. Generative music is unpredictable, classical music is predicted. Generative unrepeatable, classical repeatable. Generative music is unfinished, that's to say, when you use generative you implicitly don't know what the end of this is. This is an idea from architects also, from a book called How Buildings Learn, the move of architecture away from the job of making finished monumental entities toward the job of making things that would then be finished by the users, constantly refinished in fact by the users. This is a more humble and much more interesting job for the architect.

Generative music is sensitive to circumstances, that is to say it will react differently depending on its initial condition, on where it's happening and so on. Where classical music seeks to subdue them. By that I mean classical music seeks a neutral battleground, the flat field. It won't be comfortable — with a fixed reverberation, — not too many emergencies, and people who don't cough during the music basically.

Generative forms in general are multi-centered. There's not a single chain of command which runs from the top of the pyramid to the rank and file below. There are many, many, many web-like modes which become more or less active. You might notice the resemblance here to the difference between broadcasting and the Internet, for example.

(I should probably read that Kevin Kelly book, and How Buildings Learn.)

What will be the new metaphor of LLMs?

I think what artists do, and what people who make culture do, is somehow produce simulators where new ideas like this can be explored. If you start to accept the idea of generative music, if you take home one of my not-available-in-the-foyer packs and play it at home, and you know that this is how this thing is made, you start to change your concept about how things can be organized. What you've done is moved into a new kind of metaphor. How things are made, and how they evolve. How they look after themselves.

“Metaphor” is another interesting word for this whole bundle of thoughts. The back-and-forth of language and sensory experience (key to multimodal art).

“Parlor trick” is also a classic term of derision for toy-like things like the current applications of Midjourney and DALL-E. But who has written about literal parlor tricks?

There’s a very interesting book by Lakoff and Johnson, that famous thirties singing team, it’s a book about metaphor, it’s called Metaphors We Live By. They give a very clear example of the effect of metaphor. They say we use in our culture the metaphor, argument is war. All of our language about argument “she defeated him”, “he attacked her position”, so on and so on, they are all arguments that relate to fighting.

Well, now I need to read Metaphors We Live By.

What I'm saying, I suppose, when I talk about these things here (on his chart of the differences between generative and classical musics), I'm saying we're saddled with a whole set of metaphors that belong over here. Those are our metaphors about how the world works, how things organize themselves, how things are controlled, what possibilities there are. Generative art in general is a way of not throwing those out, we don't get rid of old metaphors, we expand them to include more. These things still have value, but we want to include these things as well.

My feeling about artists is that we are metaphor explorers of some kind. ... An object of culture does all of the following, it innovates, it recycles, it clearly and explicitly rejects, and it ignores. Any artist's work that is doing all those four things and is doing all those four things through the metaphors that dominate our thinking.

Done!

Time to grab some books while I’m thinking about them.

Metaphors We Live By: check.

Out of Control, by Kevin Kelly: bookmarked as a maybe for the future.

From New Book of One Hundred and Fifty Parlor Tricks and Games from 1905, there’s a trick called “To Name all the Cards in the Pack in Succession” that opens:

This is an old trick, but a very good one. To perform it, you must arrange the cards of a whist pack beforehand, according to a given formula, which forms a sort of memoria technia. There are several used, but all are similar in effect.

Feels like it could be the epigraph of a piece.

Searching for “memoria technia,” it looks like the usual form is “technica” and that’s just a typo. It’s basically a mnemonic device.

The root word of “mnemomic” is “mnemon,” Greek for “mindful.”

Parlor tricks are definitely cozy.

I should pull up some highlights from all the coziness books I read a while back.

3:26pm

I am clearly still resisting “really writing,” but I did have an idea for a very plausible procrastination technique: reading back through this whole log so far and noting down what lines still spark as insights, then taking a little break to see if it all integrates.

I MUST break the seal on “really writing” today, and I will be a little sad if it’s not 1000 words. But a feeling of light failure after significant effort is not the worst thing in the world for motivation, as long as the seal has been broken. (Otherwise I feel like a really terrible student.)

Loose ends I’m uncovering:

• Spend more time in the #philosophy channel on the Midjourney Discord—I’m sure there’s more there

• Things I must cite in this piece: Robin’s NOTES ON A GENRE, Brian Eno’s diary, Stephanie Burt’s fan fic piece, Neil Gaiman’s genre piece

• Reach out to Richard Moran

Insights:

• Post-AI art as being about what AI can’t do

• Commercial illustration as a corollary for AI-generated images

• “Imagine” as a recurring theme—the command, the conference, etc.

• Career of prompt designer—collaborating with AI. “Collaboration” as a verb for working with AI.

• Definitely something about Kid Pix, mark making, and wacky brushes

• With prompts, the recipe is right there in the caption. Explicable creativity.

• Captions, including: when you download the output of these models, the prompt is right there in the file name

• The impulse to re-trace these images by hand in mixed media

• Elegant simplicity —> interesting complexity (generative music); massive complexity —> simple images, many of them repetitive genre work

• “I think everyone should have to generate images in a style they personally like before giving up on image generation.”

• “Looking at other people’s images is like looking into other people’s subconsciouses—a little unseemly.”

• Generativity is a deep value of mine—the ability to generate lots of plausible ideas but not hold any of them tightly. Variation and selection (what was the phrase from The Beginning of Infinity?)

• The observation that these images are perfectly serviceable for editorial illustration

• The journey through dismissal: making it through a thicket of meme-inspired, steampunk-inspired images.

• Yuletide in the fan fic universe

Scenes:

• Sitting down at The Interval for the first time. The artwork over the bar keeps shifting.

• Huddling with Erik around a laptop screen for the GPT-3 brainstorming session. (How I wish I had those notes!)

• Writing poems for alloy.com

Words:

• Improvisation / improve

• Imagination / imagine

• Surprise

• Metaphor

• Parlor trick

• Genre

• Gradient

• Iterate

• Drawing board

Images I liked:

TOO MUCH already, my brain is really fried. I gotta take a break if I’m going to break any seals at all.

4:28pm

I made some progress in a fresh doc. Only about 500 words worth, but at least the seal is broken. What eventually worked was “interviewing myself” to get basic ideas on the page.

I AM SO TIRED! I can’t wait to see what bubbles up once I’ve had a chance to sleep on all of this.

7:51pm

Our perspective on these tools is clouded in at least four ways: lack of access, lack of imagination, feelings about art, feelings about genre

• Slippery susceptibility to “well, if you put it like that” (see: calling the images “illustrations”)

• If children love it, it will be important—and they do, so it will

• Probably this piece should be a series of seemingly separate but then interlocked passages

August 13, 2022

9:40am

It’s the weekend, and I debated whether to spend any time here—it seems logical to protect at least one day a week for my mind to relax. But I figured that if the alternative was “do nothing,” I could find a way to let two twenty-minute bursts here “be nice”—do the “nice,” not-too-hard version of them, and most of the value is in putting background threads to work in my brain. To “touch the keys”—I forget where I first heard that phrase, but it’s the thing of touching the keys when you walk past the piano in the house to remind yourself that playing is possible.

So where am I at? I feel pretty overwhelmed about “turning this all into something,” and equally worried that I’ll have put in all this thinking and end up with no publishable material to show for it. My mind immediately goes to how to radically change the terms of engagement—should I just publish a lightly-edited version of this journal?? (After all, that’s what Brian Eno did in the end.) Should I not expect something grand and just publish like a month of daily micro-thoughts instead? But none of that feels right until I try to get my thoughts straight. And I do (for now) have another blank day coming up on Tuesday.

Something haunting in the back of my mind: I am of course, due to my job, hoping that writing and publishing this helps me do at least one instead of zero deals this year. I worry about a fear-powered act of creation, but it’s not really pure fear—I have some faith, too. I think I’m mostly doing the right things in my work, and this is one of the right things, and I enjoy it in its own right and also pressure helps a bit in getting words on the page. But, honestly, the pressure of getting words on the page is a bit nicer to think about than the pressure of doing deals, because at least it’s more in my control.

Anyway, back to the main thread. If I think back to what I know has worked for me in the past, it’s selecting and then arranging the quotations I want to use. This is what I always did in college: assemble my excerpts, print them out, cut them into strips, then rearrange them until I felt happy. And then the act of writing felt mainly like composing connective prose. And most of the thinking was in the spark of identification (this quotation, not that one) and the intuitive arrangement of them—but all the steps felt easy in their own right. So I think one good task, since I know from Thursday that there are certain pieces I want to cite extensively, is to devote some time today to going back to those pieces and pulling out the clear excerpts. It feels like it’s “nothing,” just copying and pasting work, but if I completed it I feel confident I could sit down and write a bunch at once.

Wowie, Readwise says I have over 100 highlights from Brian Eno’s diary!

From the 25th anniversary introduction:

One day Stewart Brand said to me in an e-mail, ‘Why don’t you assume you’ve written your book already – and all you have to do now is find it?’, and several weeks later this way of doing exactly that dawned on me.

So the correspondence between Stewart Brand and Brian Eno is in general a big inspiration to me and in particular an inspiration for this document—I just love the idea of choosing your thinking partners and having very thick correspondence with them.

You can see further evidence of how inspiring I found it in this bit:

My conversation with Stewart Brand is primarily a written one – in the form of e-mail that I routinely save, and which in 1995 alone came to about 100,000 words.

I write to Lisa about my life nearly every day and I noted a few months ago that it came out to about 30,000 words over the course of a month, so probably it’s more like 300,000+ over the course of a year. But same order of magnitude.

Okay, this I can definitely use for something about Midjourney:

Simple rule — choose the right four colours in the first place and everything looks fine.

It converges on blues and oranges! Maybe I can go look for a Mad Max bit while I’m at it. Ooh, what a find! Here’s an interview (on YouTube) WITH THE COLORIST of Mad Max! MAD MAX Colorist Breaks Down his Grading Process The auto-generated transcript is a little hard to read on its own, but maybe I can listen to the interview as a podcast later while I’m in the kitchen. (Goals for today: ginger tahini spinach dip from Snacks for Dinner, double batch of almond flour cookies for the freezer for breakfasts, eggplant mini pizzas. This reminds me: one of my favorite recurring observations is that very “different” foods end up with similar nutrient compositions—mac & cheese is nutritionally similar to pizza, for instance. Frappuccinos are nutritionally similar to milkshakes. Kind bars (coded “healthy”) have similar nutrient compositions to the sugar cereal “bars” that are coded as a healthy alternative to candy bars—but they’d all look similar nutritionally. Anyway, almond flour cookies made with maple syrup are nutritionally quite similar to gluten-free waffles with maple syrup, which I wouldn’t blink at having for breakfast, so I decided to just go with the more portable form factor—plus, I get a jolt of the feeling of “indulgence” because cookies are coded as “treats,” it’s just the most nutritious form of them I’ve come up with.)

Ah here’s a good-seeming written interview with the Mad Max colorist, written in the first person: https://lowepost.com/casestudies/mad-max-fury-road-r9/

Every time I worked on a shot, I kept saying to myself "make it look like a graphic novel".

It’s almost like the colorist’s giving himself DALL-E-style prompts.

Still struggling to find the original Mad Max article I remember, but I did find a good entry point to orange-and-blue in films in general—the links here: https://www.criticker.com/forum/viewtopic.php?t=5025

One of them is this Priceonomics post from 2015: https://priceonomics.com/why-every-movie-looks-sort-of-orange-and-blue/ It quotes this TV Tropes wiki page: https://tvtropes.org/pmwiki/pmwiki.php/Main/OrangeBlueContrast

While other pairs of complementary colors are associated with specific concepts, fiery orange and cool blue are strongly associated with opposing concepts — fire and ice, earth and sky, land and sea, dawn and dusk, invested humanism vs. elegant indifference, good old fashioned explosions vs. futuristic science stuff (and the obligatory Good Colors, Evil Colors contrast). It's a trope because it's used on purpose, and it does something. Especially in theater, orange and blue are sometimes referred to as amber and teal in this context.

Now this is exciting because it connects to what I was reading about in Metaphors We Live By. Getting somewhere!

Well, something happened to my timer and I must have paused it incorrectly, but I’ve been working pretty straight through so at this point I think I’ll just go to 10:30am on the clock. Maybe back to the well of Brian Eno’s diary…

Not related to this piece, I don’t think, but I loved this remark:

Value the ears that things sound good through.

Very connected to Julia Cameron’s idea of a “believing mirror,” or the idea of a “first reader.”

Again, not related, but so poignant for me:

Perhaps cynicism is not a containable talent – and ends up extending to oneself.

“Containable talents” vs. “incontainable talents”—which are which?

This makes me laugh out loud:

Finding it hard to read my smaller writing these days: must think about reading-glasses (chance for a general face rethink).

Chance for a general face rethink! I just love it.

Ooh this might be good:

I mean, is ‘mediated vision’ the price of imagination?

What passage was that from? Ahh, back to the source—it’s a passage about autism. I think the state of the art on autism has advanced significantly since 1995, but basically what he’s saying is that people with autism can sometimes see things without the filter of concepts—a literal view on the world is in some ways a clearer, unmediated view of the world. But imagination is not literal.

This is another part I love:

Acknowledging that a computer is actually a place for sticking Post-it notes (mine is surrounded by them), make the frame much bigger – give the conceit ‘desktop’ some real meaning. The problem with computers is that they exist too exclusively in the electronic domain: what you need is a transitional area round the edge.

This is maybe advice for this writing project, but also contains a reference to “surprise” that might be interesting to juxtapose against all the other surprise passages I’ve dug up:

More and more I find I work better with quite strict structures around me. What I was doing last week in those early mornings was working up some new material for these Bowie sessions. I only had a few days – and the effect of this is to focus attention. Less exploring of all the possible journeys you could make; more determination to take one journey (even if the choice of it is initially rather arbitrary) and make it take you somewhere. The big surprise for me when I work like that is discovering myself capable of an almost ‘automatic writing’ way of working. I cease to evaluate much, instead just letting something carry me along. Listening back later, I think, ‘How on earth did I get an idea like that?’ Working with greater leisure, my ideas become much more ‘reasonable’ and surprise me less.

This is where I got the idea of a “bureaucratic day”:

At the studio, a bureaucratic day. We finished the mix of ‘Dummy’ but then had to spend hours doing the mono version and the four-channel version and the version without vocals and the four-channel version without vocals and the etc., etc., etc. This is why I hate working for film – the clerical work is overwhelming.

Ever since reading the diary, I’ve been thinking a lot about how even the artiest artists have clerical work to do. Brain Eno even had a manager to do most of it, and there were still bureaucratic day! It’s just not something that can ever fully be gotten rid of. I do have an inner bureaucrat that takes pleasure in that sort of order, fortunately. Ohh, that reminds me of the bureaucrat book—The Beautiful Bureaucrat, was it? Let me see if I can dig up some passages from that.

Yes, it was The Beautiful Bureaucrat by Helen Phillips. And, fortunately, I do have highlights! Thank you, Readwise. Thank you, past self.

“I just want to feel immaculate for a few minutes a day,” she said.

Ohhhh, me too.

She wanted to jog down the hall, but she made herself walk the bureaucrat’s walk, the weighted scurry.

The weighted scurry!

Sometimes, in the depths of the afternoon, Josephine would have a thought—an intense, riveting thought, incongruous with her current task and location, something she ought to share with Joseph, a hint of a scene from a dream or a forgotten memory from when she was a kid, a complicated pun or a new conviction about how they ought to live their lives—but then the moment would pass and the thought would be lost, trapped forever between the horizontal and vertical lines of the Database.

Aha, and SLOAN AGAIN: he blurbed it!! I had a feeling he had something to do with how I found out about it. https://www.helencphillips.com/the-beautiful-bureaucrat

The Beautiful Bureaucrat reads like Borges in Brooklyn, with its cerebral pleasures buttressed by Helen Phillips’ precise, resonant depictions of love, marriage, sex, and terrible apartments. It bends from uncanny to unsettling and ends at very deeply satisfying.

What’s Helen Phillips up to these days? Sounds like she has a new novel, HUM, coming out in 2023: ​​https://www.helencphillips.com/bio And here in her bio: “Helen has received a Guggenheim Foundation Fellowship, a Rona Jaffe Foundation Writer’s Award, the Italo Calvino Prize in Fabulist Fiction, the Iowa Review Nonfiction Award, the DIAGRAM Innovative Fiction Award, and a Ucross Foundation residency.” Could “fabulist” have anything to do with what I’m writing about? From https://en.wikipedia.org/wiki/Category:Fabulists

Fabulists are authors of fables, in the normal sense of "a narration intended to convey a useful truth".

A narration intended to convey a useful truth—that seems poignant. “Useful” is related to “pragmatism” here. And it’s all related to “folk tale voice,” from The Gift—language from the edge of sleep. The possible connection to generated imagery: it is a sort of collective dreaming, and also the images themselves can be quite pragmatic—fit for a purpose.

It’s 10:32am now, so one final thought I wanted to make sure to capture. I was thinking back to the idea of “total authorship”—maybe the exact language Robin once used, but maybe not. Anyway, his point was that novel-writing was one of the few remaining creative fields where a single imagination could make the whole thing without collaboration—build a whole world that reassembles itself in other people’s imaginations. Movies are big productions! There’s overhead. But a novel can be written by one person, start to finish, and have total cohesion. Well, here’s what’s interesting to me about generated imagery: it could be an extension of “total authorship,” where the de novo art is in the novelist coming up with the imagined world in the first place, but then the illustrations are their way of expressing their idea into the model until the model returns something close to what was in their imagination. Is that more or less “total” than collaborating with a human illustrator who might be able to take your idea places you didn’t expect? The model can, too.

August 14, 2022

8:36am

Playing around on my phone while lightly watching Bentley. She used crayons this morning to color all over some lavender construction paper and then asked me to use safety scissors to cut it into a dragon. She’s also been very into her tiger stuffed animal this morning. So I decided to give a prompt to DALL-E on my phone, and got this back:

I think these tools are going to be so powerful for communicating with children.

10:22am

Something I don’t want to forget: the thought that aesthetic consistency across every prompt is going to be valuable—a “house illustration style” that can be tuned. Ideally at the level of hex codes, even.

Another observation: it will be amazing when these engines can spit out Figma or Photoshop files with editable layers.

August 21, 2022

2:54pm

I just noticed that the text of the past few entries flipped back to black! I guess because I typed a few stray thoughts on mobile. Now back to purple.

It’s Sunday afternoon and Erik’s out with Cooper on a scooter jaunt. Meanwhile, Bentley went down for her nap about an hour ago. I was going to say “just went down,” but then I looked at the time and that’s not true at all; it took me a full hour to gear up. Sigh! I guess it just is what it is.

Sighing is of the moment: I’m at the point in this project where I feel hopeless that I’ll ever make anything of it, plus unclear about what it even is that I hope to make. It’s very common, when I’m perseverating on this, to come to the conclusion that I should just ditch the big piece and write a series of short posts on, say, LinkedIn, that capture and share the insights in a disjointed way. Would that not be better to do regularly than what’s happening right now—a big logjam on the way to sitting down to “really do it”? But then something will come in like the multi-paragraph email I got a day or two ago in response to my 9k-word MBA Decision Guide for Product Managers, which I published over a year ago but which still makes the rounds and generates new serendipity. That’s what I yearn for here, and I don’t need to only yearn; I actually know it’s possible, because it’s happened before. Basically every time I set my mind to it, it happens. So I just need to set my mind to it, right? But the big question is: when? When will I have the right energy and span of time to really take the plunge? Plus, I have some work travel coming up this week, which will disrupt all kinds of rhythms. I guess I can count on the week after next (the week before Labor Day) being a naturally slow one, so maybe that will be the moment. And then perhaps I can finish the piece over the long weekend. That feels about right. My big worry has been about letting it leak too much into post-Labor Day, when I have this looming sense that a lot of action will kick off and I need to be rested and ready. If it’s 70-80% complete by Labor Day, I believe I can get it into shape to publish for real sometime in September.

One side-effect of taking all my notes in a journal-like format like this is that I end up feeling like journaling (even when it’s full of perseveration) is a valid, related activity to “really writing.” It remains to be seen whether it’s a good warmup and leads to more total “real writing,” or whether it’s a cop-out.

Okay, 8 minutes left on this warmup twenty-minute block.

I have a sense that something good to do today would be to read back through everything I’ve written so far. That’s over (checks wordcount…) 16k words, so it will probably take a while! Maybe I can do that while I’m baking up the sriracha-puffed-rice-peanut mix I need to watch carefully in the toaster oven. (I’m in a heavy experimentation phase around homemade gluten-free snack mixes. I’ve made lentil snacking granola, maple pecan grain-free granola, Rice Chex mix, and now today this sriracha peanut one. Three of the four are from Snacking for Dinner.)

And then maybe with my remaining time I can 1) set up a table to track “deep work” sprints on this project at the top of the doc 2) generate one or two images as a warm-up.

3:28pm

Reading The Inkblots while one batch of the sriracha peanut rice puff mix bakes. A predecessor of Rorschach’s also related to inkblots, and it sounds a lot like generative images actually.

3:46pm

The sriracha peanut rice puff mix is out of the toaster oven now. 23 minutes (out of a suggested 25-30) seems safe—I checked on it constantly the final seven minutes or so, and it didn’t seem at risk of burning. (Things often bake faster in the toaster oven.) It was sort of fiddly to try to focus while also having an at-risk-of-burning thing baking in the background, so I decided I’ll wait to make the next batch until later today when I’m in kitchen cleanup mode. For now, I’m going to try to dedicate the next twenty minutes (unless Bentley wakes up midway through) to reading back through everything I’ve written here so far. I hope I can make it! It’s a lot of words for twenty minutes.

“Hermit mode”—what’s the etymology of “hermit”? “A person living in solitude as a religious discipline.” (Is creativity a religious discipline?) Ooh, I’ve never heard of anchorites / anchoresses—religious recluses: https://en.wikipedia.org/wiki/Anchorite They took vows of stability of place.

• Etymology sent me down the rabbit hole of the other recurring words in these notes, “imagine” being the big one. It made me wonder if there was ever anything like an “imagine” button in Kid Pix. Unclear yet, but I did find this amazing-looking 30-year-anniversary celebration of Kid Pix on a corporate magazine: https://wepresent.wetransfer.com/stories/kid-pix-anniversary

• Something very interesting here is the remembrances of now-artists about Kid Pix’s impact on them. This seems resonant with generative imagery:

• Julian Glander

• Kid Pix was massive for me. I’ve always been really frustrated with my inability to draw with pencil and paper as I’ve just never had the coordination or patience. I try to draw a circle and it comes out a warbly ugly potato shape every time. I try to erase it and the eraser rips through the paper. I cry.

• But the first time I used Kid Pix everything changed. Making art was fun because it followed the wacky, bright-eyed philosophy that seemed to be everywhere in the early days of personal computers. Don’t like what you did? Undo. Don’t spend all afternoon coloring it in, let the paint bucket do that! These were novel concepts to me at the time.

• I never thought of myself as “artistic” or even as a particularly creative kid. The joy of Kid pix was the broader joy of childhood — putting things together and seeing how they fit, making yourself laugh, and just knocking it all down at the end of the day. The standout feature that everyone seems to remember is the dynamite explosion that happens when you erase an entire scene. A reminder that all art is fleeting but the joy of making it is eternal.

• Justin Poulson

• My most vivid memories of Kid Pix were clicking and then shaking the mouse violently with my eyes closed for undetermined periods of time...only to open my eyes to reveal a random pattern that had been created. The ability to iterate without exhausting real world consumables provided me with an arena absent of the fear of failure. Within this digital art world I was free. I could mess something up and just erase it and start over again. Making digital art felt very much like when the scissors start to glide through wrapping paper: free of resistance.

• Kid Pix gave me an insight to the future in a simple package. I enjoyed that it wasn't perfect because it meant that I didn't have to be either. It was okay to make mistakes, and sometimes the mistakes led to the best-looking art. This encouraging environment is what made this program a “gateway drug” to the image processing programs of the future that I still work with today in my professional practice.

I was hoping there would be a quote with the word “imagine” in it so that I could use it as an epigraph, but I guess I could just search for “kid pix imagine.” Nothing jumping out at me in the search results, but possibly something to return to later.

4:17pm

So many words, yet still no images generated today! I guess time to rectify that. I did actually generate one image this morning; Cooper asked about a robot / alligator / treasure chest scene and, fortunately, I had the presence of mind to generate an image instead of searching for one. Actually, generating images doesn’t feel so different from searching for them—and indeed they are kind of the same act, one is just navigating latent space and the other the public web. But the latent space is one drawn from every image on the public web. And this also points me in the direction of finding a good piece to reference on latent spaces.

The first piece I found turned out to be quite good: https://towardsdatascience.com/understanding-latent-space-in-machine-learning-de5a7c687d8d

Also touching: that same Medium publication is the one that some of Daniel Ross Fein’s pieces have ended up in!

One interesting question, then, is how is generating images not like searching. I think it mostly is like searching—it’s more like searching than not. But it also has a flavor of true surprise: chances are, you’re the first person ever to see this image. So it also carries a sense of discovery. How does our conception chance when generativity is about discovery rather than creation? Both contain excitement, both contain utility—but one is about mark making, and the other is more about imagining.

Just bopped over to Discord to load up Midjourney, and noticed the splash page. “IMAGINE”—I think “imagine” is a key verb of the moment.

Something I’ve noticed: I feel compelled to watch as the Midjourney bot resolves my images, from 0% inching along to 100%. Almost like if I don’t watch, it won’t turn out. And also like making photographs in a darkroom. I suppose photography had some of these same challenges—you’re not “making marks,” you’re just “capturing” (/“discovering”) the world. Yet there’s definitely art in what you notice and how you capture it, how you frame it. So now we’re back to generative imagery as art.

I actually really like some of these:

Trying to figure out the right book about photography to read. I remembered Susan Sontag’s On Photography from my college course about photography, but reading the Wikipedia article makes me think it won’t be quite right: https://en.wikipedia.org/wiki/On_Photography

But the idea that’s coming to me is to treat generating images as photographing our imaginations through a specific lens (model). AND, it’s interesting that this lens always (in the current expressions of the tool) comes with “options”—a grid of four at a time. This really impacts the experience, it anchors the entire experience in abundance and flexibility. And reminds me of the Midjourney CEO’s observation about a river of responses to a prompt—it’s water.

Does DALL-E show a grid of four even when you say “surprise me”?

Yes, it turns out “Surprise me” just primes the input box with text—you still have to click “Generate.” And I think this is the first time I’m noticing that the button says “Generate.” “Generate,” “Imagine,” “Surprise.”

I just “made variations” of the one I liked and I notice I’m much less interested in the variations (slight differences) than in the original experience of discovery.

“Search” and “Discover” are often paired as verbs at feed companies. “Search” is seeking something you know exists (without necessarily finding it), “Discover” is finding something you didn’t know existed. “Find” is sometimes floated as an alternative to “Search,” as a verb.

4:44pm

I notice how quickly my brain gets tired doing this deep thinking; another argument for spreading it across days. Bentley happens to still be sleeping, so I think I’ve got another twenty minutes if I want it, and I do. I can return to just reading back through this journal from the top.

Re-reading, one loose end I hadn’t chased down was about the gradient trend in graphic design. This piece is anchored in 2019 and says the trend revived in 2018: ​​https://99designs.com/blog/trends/gradient-design-trend/

Another thread here: curation / Tumblr—and Tumblr was/is also a nexus for fan fiction.

Genre / generate — seems like they must have shared etymology, but a quick search is not conclusive.

To follow up on: the history of commercial illustration.

Caption is an interesting word. Same root as “capture,” but my reading is indicating that it’s a roundabout thing—“caption” was borrowed from property law, where “seizing” has a much different meaning. Still, neat that you “capture” a photograph and then “caption” it. And you “imagine” something and then “discover” it in latent space.

5:29pm

From The Inkblots…multimodal creativity!

Ooh this feels a lot like latent space:

Similar to compression:

Aesthetic / The New Aesthetic. I didn’t know “aesthetics” was a field of psychology.

Whoaaaa

This is quite wild. The process of compression / condensation of the unconscious has a lot to do with latent spaces.

9:19pm

I was thinking back on The Power of Glamour as a powerful reference text on images, and decided to dip back in.

Focuses and intensifies a preexisting but previously inchoate yearning! Does generative imagery do that?

August 28, 2022

11:02am

It’s 11:02am on a Sunday and I have declared this Writing Day. I definitely feel some nervousness; what if I don’t have enough time to write all that I have in mind? But that is bound to be the case: I have too much in mind, always. The opportunity of the moment is to channel some of that thinking into a compact set of prose. Shorter is better…unless longer is better, to create more self-selection in who reaches out to me in the end.

I’ve given myself five minutes (on a timer) to warm up into this project right now, but after that it’s on to the “two twenty-minute blocks per calendar hour” regimen.

I spent some time this morning trying to counter my fatalistic feeling about “when will I EVER have time to write?” and my other fatalistic feeling “what if this project journal is just a DISTRACTION?” by lounging on the couch while lightly supervising the kids and re-reading this whole doc on my phone. This gave me lots of ideas! I took some notes along the way.

• Tools for imagination intro—tools for thought, but thoughts are words

• Remember to mention that I’m an investor, gift of talking with people before anything’s documented

• As of this writing, here’s where we’re at with models

• And here are the inputs to my mental model: the founders I’ve met, all the books I’ve ever read, the conversations I’ve had—but not an exhaustive search. By writing this piece, I hope to open many more conversations. This is my prompt.

• It began to feel like this connected to everything I’ve ever cared about (then callback at the end to Rorschach)

• Brian Eno scene at The Interval

• Read the Diary. Multimodal art, generative models, but also just journaling—in that spirit I’m interested in a journal that illustrates itself. (Beacon: a journal that illustrates itself)

• Aesthetic…blue orange, names of other aesthetics. Beacon: controlling the aesthetic in a narrow band. (Art direction.) Color clashing. Everyone should have to generate in a style they like. Take an image, describe it in words using a model, then extract all but the aesthetic words and sub in more.

• Art and artlessness.

• Kids. Kid Pix, Mark making, kid pix professional, using it with my kids.

• Genre, fan fiction, extending worlds. Familiarity is cozy.

• Images are harder for most people to make

• Love the list of use cases, who wrote that?

• Limitations: light source, guardrails for controlling aesthetic, words in images

• Images as communication: responses

• Shadow side of imagination

• It feels like searching instead of creating. Latent space, Brian Eno quote—all you need to do is find it

• Generating these images is like searching for them

• Feed in images (per Tumblr / Pinterest says), use various distancing techniques to generate a feed of images. For instance, converting to words and then mutating those words.

• Slipperiness of verbs: generate vs create, trying to put it all in the whimsical / aesthetic realm to avoid the shadow side

• Images as reward

• The good feeling of knowing what you like

• Request for software: Kid Pix reinvented with generative imagery

• Increasing surprise: hold most inputs constant and then vary one

…and then a few more stray notes in an email sent to Lisa

• Free association

• The way the imagination works

• Daisy chaining tools together

• Each tool a little unpredictable

• Semiotics

My heart’s beating faster! Time’s up, time to shift into writing mode.

11:29am

One twenty-minute block complete. Giving myself a five-minute journaling break now that I checked my phone for any urgent messages. I basically only wrote the first sentence. But I did a lot of thinking work to select from among all the things I could have written. One slight shift I played with, after reflecting on how much I enjoyed seeing this pale blue doc in document previews, was to get more serious about the semiotics of typefaces / how they’re coded in my brain. It’s Times New Roman for me. (In the draft doc.)

I feel excited about the “tools for imagination” insight (language intentionally paralleling “tools for thought”), but I’m worried I’m not the first to come up with it. I’ve decided to just write as though I did, then research any parallel provenances later. It’s both reassuring and disappointing to me that this is not actually a college term paper—I have more than just today to write it. I wish it were realistic to make it just today so that this could be over faster. But I think it will be better if I make urgent progress and then also give it time to marinate. But not too much time! I’ve essentially concluded that it would be good to publish it this week, since I think there will be a big backlog of announcements that people feel like making right after Labor Day and I don’t want to get lost in the shuffle.

Okay, five minutes up!

11:57am

Another twenty minutes down, and another five to journal. This is going painfully slowly, but I know some of it is that I’m building up an architecture for the piece in the back of my mind—figuring out the voice, identifying the beats, choosing my verbs carefully. This is how it always goes, I think. I just wish I had all day; the kids will be back home from the park with Erik in maybe ninety minutes. That gives me time for three—maybe four—more twenty-minute writing blocks.

I feel / fear this will take twenty hours to write (which would be sixty twenty-minute blocks), but I suppose that’s something I can just count. I mean, if I get in six writing blocks today, that’s 10% of the way there, and probably the middle 60% will feel like being on the glide path—once the edifice is put together, and before it’s time for detail work.

Time’s up!

12:24pm

I feel like I’m finally getting somewhere—time is starting to go faster, and I’m starting to get excited because I can feel that this piece will be beautiful enough. It’s funny: I spend so much time in journal-esque writing mode / correspondence / “communication” more generally that I miss the part of my brain that intuitively makes things more lyrical. It takes longer, it’s less stream of consciousness (“stream of consciousness” is a good phrase for the piece, actually), but it satisfies a part of me that cares about beauty and cares about being the one to make it. I am much better at creating beauty in words than in any other format.

I am very happy with the fact that capturing this stream of consciousness has given me a way not to lose my place. (Google Docs wanted to autocomplete that to: “given me a way not to lose my place in the world”—quite poetic.) It seems like a good adaptation of the world of writing and research to the realities of parenting, where almost all of my time is aggressively chopped-up. And in happy creative moments like this one, I think to myself “what if EVERY Sunday was a writing / research day?” But I know enough about myself to know that the feeling of “always from now on” should only be a signal to appreciate the present.

Time’s up!

12:58pm

Let the record show that it took exactly four (4) 20-minute writing blocks in a row to feel like I know what I’m writing and like I could keep going forever. I think I found the way to turn this into a link-filled personal essay that weaves in insight and inquiry—one of my favorite kinds to read, so I hope others get something out of it, too.

I felt a sense of lightness and thrill as I ran up the stairs to grab a bag of Hu-brand grain-free cookies (Snickerdoodle flavor) I left on the kitchen counter. I’m hungry! And my heart is racing a bit, I’m sure from a combination of writing excitement and caffeine. (I tried to zero out caffeine over the past week, so anything at all now feels adventurous.) The lightness came from a feeling that, despite all the many hours of work ahead of me, I’m on something like a glide path—this WILL come together, and when it does, it will feel so good. It’s painful to realize that, when I’m not writing, I live every day with the downer sense of my own unrealized potential.

(“Realize” is an interesting verb here—makes me think of the song “Do You Realize?”)

I’m going to switch my music from Spotify’s “Moon (And It Went Like…)” radio to Shadow Planet by The Cotton Modules (SLOAN AGAIN!)

2:29pm

Erik got home with the kids and then it was time to swing into a cozy logistical interlude: make each kid a bottle of coconut water (they still drink from baby bottles sometimes because I’ve found they’re the least likely to leak, but they drink all kinds of things from them), play mind games with Bentley until she puts herself down for a nap. Cooper wanted to try the new flavor of Harmless Harvest coconut water with a “hint of Pink Lady apple,” so that’s what was in his bottle.

I feel a little sad that my thought process was interrupted, but a) that’s just life—this is where I’m at with writing right now b) I’m doing everything I can to capture every shred of every train of thought so that I can pick up where I left off even if life interrupts at unexpected intervals.

My sense about the draft right now (just under 800 words) is that it’s probably a little too personal to be of general interest, but I don’t really care about general interest; what I care about is finding my way to the five people I don’t know yet who I would be completely preoccupied with meeting if I knew they existed, which I hope I soon will.

During the pause, I felt some background sadness that I’m only now getting around to expressing what is clearly a spiky part of me, which was such a factor in Matrix hiring me. But maybe it just took until now for me to get my bearings. And, I think it’s pretty understandable that it can be hard to create enough compartments in my life to house the coach, the author, the bureaucrat, the analyst, and my inner stay-at-home mom all in every week.

Okay, the five-minute timer has been buzzing for a while. Time to get back at it!

3:02pm

It’s really feeling hard to stop typing at the end of each block now, which is a good thing. I’m six writing blocks deep today, by my count; that’s 120 minutes, or two solid hours of focused typing in the draft doc. It’s good to know this much is possible, though I suppose part of what makes it possible is that I don’t need to do much other thinking today.

Should I somehow cancel a bunch of meetings this week to be able to finish this while it’s all loaded into my mind? Or wake up absurdly early tomorrow? I suppose getting Erik’s help to get some reprieve this evening is possibly a less disruptive option in the end, especially now that the children are self-entertaining as long as we accept screen time for them. I like to think that training the algorithms, for them, is a way of developing the good feeling of knowing what they like (a phrase I’ve never forgotten from a parenting book). It also might be hollow comfort, but I suppose screen time is always a matter of degrees.

I’m worried right now that the piece is not only too personal, but too basic. But I suppose I would feel that way six writing blocks in. I have probably fourteen to go! (Based on my estimate of twenty total.) I have such a sense of scarcity around focused writing time in my life with young kids and a busy inbox, which is a funny feeling to jut up against the abundance of what’s possible with these models.

3:30pm

Those twenty minutes flew by. Before I know it, Bentley will be up! Let’s see if I run out of steam first.

One interesting thought is that Sundays might be so optimal for writing (I need a day to re-regulate after the workweek) that actually the thing to do is to push all chores to Friday/Saturday or the workweek itself. Tuesdays are fine for research, but I’ve still found it hard to write then. But, who knows; it could just be that I’m at the right point in this project to finally be humming. It’s funny and a little sweet to me that as soon as I’m experiencing something working, I want to congratulate myself and make a template of it. As though I could know where it would lead! Although I suppose “feeling good” is in itself enough of a novelty that it’s worth paying attention to, and valuable in its own right. I certainly feel good in a different way when I sit down to write for real.

Time’s up!

4:00pm

This time I really did run over the time. And now I can feel my brain starting to get tired; eight blocks may be about my limit. (Or, maybe I’ll feel another surge of energy this evening. Or wake up early tomorrow and get back at it.) If I really had a term paper due tomorrow, I guess I would just push through—that’s what I used to do. But I would also pay for it soon thereafter. Ah, I wish I could just be done already! I so badly want to publish the piece this week, but I think I probably need to wait until it’s actually done.

So far, my epigrammatic quotes are not playing the role I thought they would.

Patience, patience with myself! I’m only 8 blocks in, and I always (“always” since this morning) thought it would take at least 20. I’m not even halfway through.

At least doing chores will feel like a break after this. And probably Bentley will be up after her nap after one more writing block or so, so I should make one more push and then leave myself somewhere to begin next time. Maybe this block can be about outlining the rest of the piece while it’s all loaded into my mind.

4:30pm

Okay, final twenty-minute writing block (for now) complete. I focused on adding headers for the next few sections I have in mind and selecting epigrams. I still worry that this will be too basic, or heady, or dreamy, or something like that…but I think it will still be enough to start the kinds of conversations I hope for.

I made it through 9 20-minute writing blocks today (180 minutes, or three solid hours of writing) and ended up with just shy of 2,000 words in the draft, if we include epigrams and headers. I think it will probably be 4-5,000 words in the end.

It’s hard to be coming out on the other side of this protected writing zone; I feel like I still have so much to do in life, in my inbox, in everything, and it’s easy to feel foolish for pushing it aside for a bit. But doing something like this (figuring out what I really think; reconnecting with creativity; getting ready to share it broadly) is one of the highest professional impact and personal satisfaction priorities I can think of, so maybe instead of feeling foolish for leaving so many loose ends in life, I should be trying to leave more loose ends so that I can write more.

I still haven’t searched for “tools for imagination”—I remain pretty scared that it’s already “taken,” and I want to try to express more of what I mean by it before I get dejected.

This will have to be enough for now!

August 29, 2022

3:49am

Well, it happened: I woke up at 3am. I set some alarms for early (5-5:25am), thinking that would give me maybe an hour or a little more to write before the kids got up. But it was always a bit of a trap; an hour would have been barely anything (in fact, it’s taken me nearly an hour just to gear up after spontaneously waking up). It was just that 5:15am was the earliest that would have still barely gotten me a reasonable amount of sleep (6 hours). But it often happens that my body hears “wake up early” and then overshoots out of excitement / nervous energy, and that’s what happened here. I just hope I’m okay in the end; it’s a busy workday and I have a feeling I’ll be zonked by the end, and I need to be in a good place in time for a quick work trip this Wednesday. On the other hand, I do seem to wake up absurdly early about once every ten days anyway, since I think eight hours is a little too much for my sleep needs but maybe it’s the amount my per-night sleep cycles pop out at.

I’m giving myself ten minutes here for gear-up journaling, and then I think it’s time to make the most of this wished-for-but-overshot morning time. I should have 2.5-3 hours before the kids wake up, which should be enough time for about six twenty-minute writing blocks. Maybe seven. It feels like I should get more of a payoff for waking up so early! But maybe six is okay; my brain was pretty tired after just nine blocks yesterday.

My draft made it to nearly 2,000 words yesterday, and I’m hoping to end this morning’s writing experience at at least 3,000 words.

4:19am

I took the first twenty-minute block to re-read a bunch of yesterday’s writing and poke at it—fixing up some words here and there, toning down some grandiosity, having realizations about needed structure and further section headers. It was actually nice to have some loose ends to poke at, so I should remember that multi-day writing rewards leaving little tasks for the next time, since it’s a good warmup and unnecessary in the moment. Opening browser tabs to add links, for instance.

My laptop is starting to run out of batteries, so after I wrap up five minutes here I need to grab a charger from the other room before the next block.

I would say that the intro feels very close to right and the first substantive section is still sort of a mess, but I think I just need to write a lot more total so that I have more puzzle pieces to move around. I would say my estimate of twenty twenty-minute writing blocks seems wildly optimistic at this point. First of all, that’s only about seven hours of active writing—I just hadn’t done the math. Second of all, if I do manage six blocks today, that will take me to 15 total between yesterday and total, and I don’t feel like I have anything like a line of sight to being ¾ done by the end of this morning. But it will be so interesting to figure out how long it really takes! And, it should be said, I’m enjoying every moment anyway; I was in a much better mood yesterday evening because I’d taken so many hours during the day for my own creative expression. I even played more actively and imaginatively with my near-preschooler, Cooper.

4:37am

Mid writing block, but quickly, a loose end to return to: see whether the models respond to hex codes.

4:51am

Another writing block down, this time focused on the Aesthetics section. I’m glad I’m writing it, but I have to admit that it’s actually somewhat depressing to write. Seeing the promoted sample images again (vs. the models’ responses to my own imagination) reminds me of what turned me off from all of this in the first place—hyperrealistic whimsy, genre, an abundance of photorealistic fantasy women (this really gets me down), a Lisa Frank color palette. It’s such a bummer (to me)—these aren’t my aesthetics at all. The models are capable of so much more, but these images are also a mirror: these are what some other people like or think is impressive. I don’t need to share their aesthetics for the models to be useful to me, but without seeing counter examples, I might never try.

5:22am

I am moving very slowly through the aesthetics section; I guess I have a lot to say. And that’s making me think that this piece overall is going to take forever to write. BUT: I have to say it feels really good to be in the more mechanistic part of writing where the structure (or “a” structure, enough to liberate me to get words on the page) sets the tasks. It feels bad to remember how much I dislike the default aesthetics (oh, “default” is a good word to remember here) used to showcase these models. But I’m already in deep enough that I should just articulate that. And, anyway, I’ve proven to myself that other aesthetics are possible.

5:54am

Still in the aesthetics section. Good grief! But I am doing some good thinking on the page as I articulate what I mean. And I maintain that it’s okay for this piece to be very, very long; a desirable outcome is for there to be large dropoff and high self-selection in who actually reaches out. (As long as anybody actually reads it in the first place.) And I think having “something for everyone” supports people who enjoy my writing for any reason having enough to react to that they’re able to find the surprise to share.

It’s getting late enough in the morning that I should probably transition upstairs to be there in case Cooper gets out of bed. He often wakes up in the 5-7am zone and comes into our bed to cuddle and then falls back asleep, which is a sweet pattern that I’m fine with. But if I’m not there, he gets agitated and then it’s harder for him to get back to sleep. It’s fine, I nearly always do long-form typing lying down anyway, so doing it upstairs isn’t much of a hurdle.

I’m really glad I’m up and making this progress and seeing how much further I have to go, but I have to say I feel quite intimidated because I have no idea how I’m going to get enough of the blocks I need to complete this piece on any kind of reasonable timeline. This week, I have this morning, then maybe two hours tomorrow morning after our nanny arrives at 8am as long as I work from home until my 10am call (but I’d need to do a quick handoff, and I probably need about half an hour of that time for crucial email), then essentially no time on Wednesday unless I wake up very early again (but I’m not sure that’s wise, since I have a flight that evening and a board meeting Thursday morning), then no time on Thursday unless I use the flight back (and I have to imagine I’ll be too tired), then maybe a lot of time on Friday but my brain will probably also be quite tired at that point. And some time over the weekend, but we have a goal of making it a family weekend so TBD. But! I’m doing what I can. And maybe the aesthetics section is just naturally a very long section since I have a lot to say.

6:13am

Update: right on cue (but it may have been a self-fulfilling prophecy, since I’m sure I made some noise as I snuck back upstairs), Cooper hopped out of his bed and into ours to cuddle. I stayed there with him for a few minutes waiting for him to settle again, and now I’m back on the  main level. (He generally sleeps for an extra hour or two once he makes it into our bed.) Bentley often wakes up at 6:40am, so I may only have twenty minutes left. I would love to finish the aesthetics section, but we’ll see. I can’t believe I still have a full day ahead of me! It’s a little overwhelming, but we’ll see how it goes.

6:39am

Ugh this section is taking so long. But I am five writing blocks down, and if I can reach six, this stolen morning will have gone more or less as planned upon waking. Bentley’s still asleep for now, so I’m hoping that sticks.

I will be very glad when this aesthetics section is done enough, but it’s making me worry that it actually needs to be pulled out into a separate piece. I don’t want that! I want it all to be one thing. But it feels too long, and like maybe the point it’s making is “the default aesthetics are gauche in my view, but never fear! They can be customized!” But I actually think I need to embed enough casual judgment that then I can complicate and interrogate those reactions in the sections afterward. It’s context, how-to, but also setup.

September 2, 2022

2:34pm

I meant to write and research all day today, but it turned out I was just too tired. Traveling for work both last week and this week really took it out of me, plus a somewhat chaotic end to summer in general—trying to pack it all in. I feel frustrated, because I was kind of counting on having a big block of time today to make major progress. But I’ve known myself for long enough to know that this is just sometimes how it goes: I spend the last of my fuel, often on a combination of work intensity / preparing and recovering logistically on either side of a travel disruption / sensory overwhelm (see: taking the kids out for teppanyaki last night), and then I just have to spend a while feeling kind of empty before I can even want to do anything again. In this case, I’m trying to be grateful that I was able to keep today open so that I could go through all the emotional phases of a return to routine in one day, less chopped-up by seeking ways to get my needs met while still being a responsible and active parent over the weekend. I think I should be able to do some writing over the weekend if Erik and I are able to divide and conquer the way we often do.

Even while I’ve been caught up in work travel, I can feel how some of the threads of this research keep working in the back of my mind. One: Sarah Guo came out with some thoughts on Stable Diffusion, etc. https://sarahguo.com/blog/foundationmodels Two: at the board meeting yesterday, our former Matrix partner, Josh, spontaneously brought up “applications of image generation” with me as a suggestion for what he’d be focused on in my shoes—and I said, aha, yes! And scrolled through some of my findings with him. It felt good—a moment of spontaneous confirmation—to feel like I’m “onto something.” Me and many people, but importantly: I’m included in many, and I can’t know how many—I can’t let that intimidate me. I just have to keep trying. Three, I caught this thread by Jessica Hische, someone I’ve always appreciated: https://twitter.com/jessicahische/status/1565728893700435969 Four, Ben Thompson from Stratechery came out with this interview with one of the Anchor cofounders, where Mike Mignano shared this about creative tools:

If you think about the platforms that have had success in empowering people to create, the reality is they do it by helping them. They give them tools or they teach them how to make really good content. We weren’t doing that, we were effectively in this first version just giving people a microphone and a publish button. We learned something though, and for the second version of the product, we threw in a bunch of creative tools. We said, “Okay, people are making stuff, there’s some sort of demand here to create content, but maybe we need to help people.” So we gave them all these creative tools, we gave them effects and things that they could do to use to make their voices sound better. We gave them very, very lightweight editing capabilities, we actually connected to the Spotify SDK and let them mix in music, we gave them stock background, we gave them all these tools. All of this was just on your phone, different than making a podcast like you and I are doing right now, all of this was happening on the phone. You could edit and put these things together with a few taps of your thumb on your phone and almost instantly the content actually was better. The content was very, very noticeably better, but here was the thing — it started to sound a lot like a podcast.

Five, I ran into a TikTok today where a VFX producer had strung together a few AI tools to make a realistic character, and it was all pretty confronting—it reminded me that not only is the “tools for imagination” line of thought probably a little idealistic, *anything* harmless might be pretty idealistic.

I got interested today in how to use image gen tools to generate never-before-seen aesthetics. I don’t want only sameness and remixes! I also want the hope of discovering something new. Maybe using passphrase generators as inputs?

This also reminds me of the Matt Levine remark about crypto: if you revere a system but want to be a part of the inception of another system, would you not feel thrilled about participating in rebuilding a bunch of the same stuff from scratch? I feel a bit that way about this stuff—I’m a words person who’s never been able to express myself as visually as I’ve wanted, but now, here, there’s a special moment in the history of images that I have a hope of being a part of.

Time’s up, but I’m satisfied that I at least broke the seal. More to come.

8:35pm
From my tabs: https://www.nytimes.com/2022/09/02/technology/ai-artificial-intelligence-artists.html

September 3, 2022

11:01am

It’s Saturday and I wanted this to be a writing morning, but it’s just going to be a little more chopped-up than that. I spent the last few hours with the kids, going on a walk all in our matching yellow rainboots (note: it’s sunny out) and then tried to get them into the car for an outing with Erik, but they basically staged a strike and refused to get in the car. So I left them upstairs with him and I’m trying to eke out a bit of time (maybe two, two and a half hours?) before Bentley’s naptime, and then grab some more time then.

First things first: I want more than a little bit of time! I feel like I could really use just acres of time. It’s so frustrating to experience time being whittled away at the margins by life stuff. I guess I can try to convince myself that time would be intrinsically chopped-up anyway because it’s too hard for my brain to work at full capacity for more than 4-5 hours a day. If I can do two hours now and then two hours while Bentley naps, I may be close to my limit anyway. And then I’ll feel so good going into an evening of life coziness! So that’s a nice thing to dream about.

It’s also nice to dream about lunch. I was feeling a little hopeless about the ingredients we have in the house, then remembered I could make a sort of nacho bar with homemade guacamole and some Papalote salsa, with goat cheddar so I can actually eat it. So that sounds nice. Long live the broiler setting on the toaster oven!

Re: the piece, I’m starting to feel a surge of hopelessness / urgency where it seems like I MUST publish next week, or at latest the week after—the field is just moving much too quickly. Maybe I should make a goal of at least posting something on Tuesday (if not a whole essay) to plant a flag that this is an area of interest for me? But I think I have to give myself at least one more day to not worry too much about what form the piece takes—I’m still in the phase of getting my thoughts out in the first place.

Eighteen seconds left on this ten-minute journal timer. I think my first step should be to re-read everything I wrote in the draft doc so far.

11:39am

One twenty-minute writing block down. I’m still struggling a bit with defining “tools for imagination.” But I have more material to work with now, so I think I need to just move on.

I really do enjoy writing. It’s pretty hard to create the right time and space for it, but once I’m in it, it definitely makes me feel more like myself.

Back on my passphrase generator line of thought. I like these! It’s such a fun warmup.

12:05pm

I have the gloomy feeling that I have about twenty more hours to go on this piece. Is it going to take all weekend? That doesn’t seem particularly fair to Erik or the kids. On the other hand, if I don’t publish this soon, I’m going to be much too late—and I think it will set up an exciting fall of meeting founders if I get it right.

September 4, 2022

5:48am

It happened! I figured out how to get some writing time this morning, here on a Sunday. I’m so excited for it. There are few things more frustrating than not being able to find writing time, and few things more satisfying than finding it at last. I think I have at least until 9am (a little over three more hours), and possibly longer.

My new thinking is that I need to find a way to chop this up and publish it in installments. I just have to. If I wait until it’s “ready” / really good, I’ll have waited too long; things are happening so fast in the field, and anyway, I’m hoping that this work can be a real factor in what finds me during fall fundraising season. (Gosh, I hope it’s a season!) And I bet a bunch of those founders will be kicking off fundraises NOW. So I have to sacrifice some completeness at the outset, which reveals the next layer of concern: that it will work so well that once the first piece is out, I’ll get TOO MUCH inbound and it will prevent me from publishing the rest and ever finishing the thought. But this line of thought can probably never be “finished”—it’s too big, and it’s all brand-new. I should probably want to be able to keep up to date. And being regular about it will probably be better for establishing myself as a magnet in the space. So, regularity it is.

In the meantime, though, I want to keep advancing the rest of the body of work so that I have more to work with eventually if overwhelm does find me. And just so that I have more thoughts in the first place! So I’m planning to devote, say, four writing blocks to advancing one of the other sections, then switch over to polishing up whatever I think will be the piece I publish this week—maybe even putting it in a blog editor so that it starts to feel more real and I start to see the real gaps.

Okay, time to begin!

6:47am

Two writing blocks down and I’m making a little progress on the section about multimodality. It just takes forever to write. I hope this turns out to be worth it! I’m feeling some relief being able to see these as interlocked pieces. It means I can go arbitrarily deep on the places I feel I have any kind of special insight or attachment. Right now, I’m making Brian Eno the anchor of the multimodality section. I should probably switch into polishing up the “Introducing Tools for Imagination” section before long, since I think that’s the most important one to get right. Probably, the origin story of my interest belongs in one post and then the proper “Introducing Tools for Imagination” belongs in the next. My heart’s beating a little faster because this means it’s coming up on time to search for “tools for imagination” and see how common the phrase already is. But not just yet.

One interesting thing about the last writing stint is that I pulled up this post to reference it: https://dianaberlin.com/posts/letters-from-the-artists-way and now I want to read the whole thing.

7:30am

Well, I didn’t expect that to take so long…but I just spent about half an hour re-reading my “Letters from the Artist’s Way” post. I don’t know why it took so long, but I am glad I did it. It helped put the present in perspective, and also all of my preoccupations—it showed me how long some of them have lasted. It also inspired me to try to bring some of the stronger life design elements of the experience into the present, now that I actually have flexibility. But, let’s just see. If I could ONLY write and adapt to the school year (with Cooper starting preschool on Tuesday), that would still be plenty. Not everything can happen at once.

I’m going to try to get back in gear for another two writing blocks before taking a shower. I really hope I can finish the Brian Eno section by then!

8:29am

Well, this is going much more slowly than I hoped or expected. I got one more writing block in over the past hour, but only one. I wonder if some real caffeine would help? I went off caffeine a few weeks ago, then briefly back on it, but suffered from terrible sleep as soon as my tolerance was gone, so now I’m trying to be back off of it for real. But it’s frustrating to not have that easy energy boost accessible to me. I’m also generally hungry; the snacks I brought down last night (so I wouldn’t have to pop back upstairs) feel like not-enough. I really want to do at least two writing blocks in the next hour and maybe by then Erik will have the kids out of the house and I can pop upstairs to grab some food.

8:53am

I was really about ready to throw in the towel halfway through that writing block—I was so tired, and I wasn’t getting anywhere. But then I decided, while I was low anyway, I might as well finally break the seal on looking up “tools for imagination” and seeing if I was actually on to something or whether it was a well-known term. WELL. Now I have TONS of energy. Turns out the phrase has definitely been used before, but the search results on Google held just a few references, all of which are complementary and none of which have anything to do with generative models.

https://www.cooperhewitt.org/event/cas-holman-tools-for-imagination-02-17-2021/

http://www.directedcreativity.com/pages/ToolsImagine.html

https://celinecondorelli.eu/selected-work/tools-for-imagination/

This is VERY exciting and makes me feel motivated to finish the initial kernel of a post.

(A thought fragment: try to find the toddler book that describes asking a toddler about food “What would make this better?” Very similar to the experience of seeing the results from a model and thinking, hmm, no, that’s not quite right…what I meant was THIS.)

I think I can transition to polishing up this first article now, confident (and feeling urgency) around being early to the concept. I should search Twitter quickly just to see, though.

9:39am

Back at it and just want to share a stray observation: the years 1995-1997 might hold the key here. Brian Eno’s diary, that Imagination conference, this DirectedCreativity book turns out to have come out in 1997…what else can we learn from that time? What else bends in on itself?

9:48am

Getting somewhere! Just wrote the paragraph about the provenance of “tools for imagination.” Now to take a five-minute break.

10:31am

Making some progress but now it’s been just a long morning total—I’ve been chipping away here for close to five hours. Breaks getting longer, brain slightly refusing to participate. But! I think I can get in at least three more writing blocks before noon, and then I will have matched last Sunday’s effort. (Nine blocks total.) And, when I put it that way, it doesn’t sound like long at all…in fact, it sounds like it will be hard for me to finish what I have in mind. So I guess that’s motivation to go faster!

September 5, 2022

10:26am

It’s Labor Day and the kids are at a park with Erik for the morning. I think this is going to be my last chance to make a big push before publishing something (“something”) tomorrow, I hope. I decided that even getting lost in the drama of “everyone” publishing / announcing things the first day “after summer,” it was important to plant a flag that this is a topic that’s interesting to me, and then I can fast-follow with more material later in the week and the week after.

It’s really funny to me how much time I need to “gear up.” Regular journaling, then project journaling, THEN, finally, get into “real writing” when I only have two and a half hours left to play with. My expectation is that I need at 2-3 blocks to load the piece into the editor, so I’m going to try to “finish” writing in 2 blocks. It’s scary! I wish I had infinite time. But I do not, so this will just have to do.

10:51am

I’m getting somewhere, but oh my word this does not feel like enough time. I don’t see any way for me to be done with writing in twenty more minutes. I’m just going to have to do my best. Maybe I can load the piece into the editor tonight after the kids go to sleep? It seems much more important for the piece to be good than to finish right this second.

11:53am

Well, three writing blocks in and it’s feeling closer, but it’s hard to imagine how just two more blocks will get me there. (Especially since figuring out images will be a part of loading it in to the editor.) I’m pretty hungry but I can eat while the kids are around but not write, so I think I’m just going to have to wait and then I can make myself some broiler nachos and guacamole as a reward. And if I do the sessions back to back, I can be done by about 12:40. Time to go!

12:22pm

Eeee!! I think I’m basically done!!!! I CAN’T BELIEVE IT!

There are still brackets everywhere, and a strong need for more images. (Can’t have a piece about generative models without images.) But the difficult creative work is through, I think. I just checked the word count and it’s rounding the corner on 2,700 words, which feels about right. Long enough that only the people who make it to the end will email me.

My heart’s beating so fast! I’m very excited.

While I’m on a high (and before the gritty march to publication begins), some notes of appreciation and gratitude:

• I am very grateful that this collaborative journaling technique (now rounding the corner on 30,000 words here) did not fail me. I hoped it would work, and in fact it worked way, way better than I expected.

• I am very grateful that Weiwei and Robin were willing to be on this doc. No matter how much or how little time they spent here, just having them on the doc meant I never felt alone.

• I am grateful to Erik for watching the kids enough that I could pursue this harebrained scheme.

• I am grateful to the kids for accepting earlier bedtimes (after protesting them for months / years) so that I could get a little more predictable quiet time.

• I am grateful to myself for taking the overwhelm of being at the start of August without a clear line of sight to a deal this year and saying, well, what can I control? I can control writing. And in under a full month, spending enough hours researching and writing that I could come up with some thoughts I’m genuinely proud of.

12:52pm

I thought the kids were going to be home any minute, so I went ahead and made myself the broiler nachos while I started a load of (long-overdue…consequence of a creative fugue) dishes. But it looks like Erik’s still at least twenty minutes away, so I’m going to break the seal on a little digital “tidying” block (10 minutes) designed for incrementally inching this thing closer to publication. Add links, fix punctuation, add images, that sort of thing. I think I can use my desire to procrastinate on home care this afternoon while Bentley sleeps as a way to add in digital tidying as a relief, especially if each block feels so small and manageable.

I’m so excited to publish this, and so nervous! I guess I should probably draft the LinkedIn etc. post text today, too?

And a digital tidying block begins!

September 6, 2022

It’s live!!! https://dianaberlin.com/posts/2022/9/6/toward-tools-for-imagination