AI companions playing with a new form of authenticity

Weeknotes 407 - Building relations with AI companions questions our sense of authenticity in new manners. And the latest captures of the news on human-AI-things-co-performances.

New forms of authenticity, according Midjourney

Dear reader!

Back from vacation! What did I miss? Will dive into that later. First, I want to refer to a sad thing from last week: the passing of Dutch writer and artist Paulien Cornelisse. I read some of her books, many of her columns, and saw many of her appearances in the media. I also listened to all the Eeuw van de Amateur podcasts she participated in over the years. It is said a lot in the obituaries; she was a master of curiosity, of wonder, of an open mind, both in a social sense and in finding the strangest things in language and beyond. It is something that everyone should strive for; I do, for sure. It drives me every week: digging through the news, finding something that triggers me, connecting it to other things, turning it around, and taking a second look. She was a master not only of that curiosity but also of sharing it in a way that engages others, makes them curious too, and makes people look differently.

Week 406: AI companions playing with a new form of authenticity

To make one connection to this newsletter. How does this relate to our relation with AI? A snippet of a conversation with Katherine Rundell in an interview was very true. It made a case that AI is not only parasiting on the work of writers, but more importantly for her, she was afraid that AI could make people shortcut thinking and learning by not feeling the need to find their own path, but jumping to conclusions, to the outcomes as presented by the AI. Losing, in that sense, a kind of human quality of life. I think that we can have AI in our lives as a tool, as a companion, as a referent in a way, but we can only do this when we bring our own curiosity, our own quirkiness, our own strangeness into that relation. We should be inspired by the attitude to keep ourselves and, more generally, our collective humanness.

It connects well to another triggered thought.

This week’s triggered thought

In de Volkskrant, writer Lucas Rijneveld looked back on stepping out of public life for a while to work as a gardener. "Ik was gewoon Lucas, dat vond ik zo'n verademing." I was just Lucas, and that was such a relief. The sentence stayed with me while I read this week's other news. It describes the relief of no longer having to think about how to behave, or about who might be watching.

That other news was about companions. OpenAI introduced dots, a few weeks after Meta launched Muse. Both are personal agents that are always on. You give them a name, they learn your preferences, and they keep working after you close the app. They are presented as butlers: book the table, sort the inbox, remember the birthdays.

The attitude behind them feels different from the agent story of the past year. That story had the human as a manager of an army of agents, reaching goals no single person could reach. Here the agent is something that stays with you. Dots feel like a form factor looking for its place in your day. Muse feels more like a relationship Meta is trying to leverage, since it sits so close to your Facebook, Instagram and WhatsApp.

Most of the conversation so far is functional. Does it book the right restaurant, does it miss an email. I am more curious about the social side. As with any personal assistant, the boundary slides. It starts with your agenda and moves on to your relations. People already ask a chatbot what to answer before they reply to a friend. When Muse follows up on a birthday for you, the message arrives. But whose attention is it?

The familiar question is whether there is a real person on the other side, and whether we can trust them. These companions turn that question around. Who am I when there is always someone along? Whether the companion is "real" or just a mirror of yourself, as the old ELIZA bot was, may matter less than the simple fact that it is there. Rijneveld's relief came from having no audience. A companion that is always on is an audience too, even a friendly one. Maybe authenticity is no longer about being unobserved, and more about choosing who is with you when you are observed. Or maybe that is too comfortable a conclusion.

In a workshop I ran with a car brand, the in-car agent was presented in the usual way: where is the best restaurant nearby? That needs to work, of course. My first reframe was to see it as a companion in the passenger seat. Someone along on a solo commute, or a third voice when two people share the ride. Not a series of questions and answers, but a conversation that keeps going. Conversation is where a lot of thinking happens. A passenger shares the view without holding the wheel. Can a companion be present without becoming an audience? That might be the real design question, more than which functions to add.

Which brings me to how this column came about. I started by thinking out loud, rambling to myself, and then used an AI to turn the stream into a structure. I had planned to try it differently this time and have the conversation with the AI while thinking, not afterwards. I did not get to it. But I keep wondering whether the thinking would have changed. Would the AI act like ELIZA, handing my own thoughts back to me in better order? Or would it behave the way it did when I only asked for a storyline, coming back with a stack of sources I had not asked for: mascots, keychains, launch details? Both change what is mine in the end. And I am not sure which of the two I would prefer.

Notions from last week’s news

In the last newsletter, at the beginning of my holidays, I noticed the most interesting part of the Apple keynote: a continuous-listening watch. MKBHD had a good breakdown of these thoughts.

I normally focus on the news from the last week, but I did a quick scan of key newsletters during my vacation. Below I dive into last week, some things I take from before: Jev, sponsored agents, model welfare, a digital fly brain, new glassholes, more rogue AI behavior, ramping up robots, EU position, secure open agents, symbiotic intelligence, connectomes.

It was a busy first day, so I did a quicker round through the news. These are my captures.

Human-AI relations

Ethan Mollick’s take on Dots and Muses. And Swarms.

How much do you delegate?

The future of software. Agent disruption. Nate explains.

AI as a religion. In different ways.

Human superpowers with AI.

AI 360 reviews.

The frustrated agents.

And AI-enhanced humans.

Hacking.

Physical AI

The physical angle of the companions is around the corner.

Drone delivery.

Open-sourcing AI gadgets.

The continuous promise of the exoskeleton.

Fighting robots, a bleak future.

To dive in later. Choi asks who the "robot maid" that tech culture keeps promising really is, and what job we want her to do.

Banning AI glasses.

Structural adaptation, relational AI, contextual shift. Potentially interesting dimensions.

Unexpected consequences of AI boom.

Tech in civic societies

The cost of a frontier AI from a leaked S1 filing.

The OpenAI culture.

The impact of AI on America society.

And on Africa…

AI sovereignty the reversed way. In two cultural angles.

And some more general news:

New Google Gemini. Anthropic robots. Books on worldbuilding. OpenAI Dev Day vibe checks. Watermarking slop. Non-Chinese open-source model.

Weekly paper to check

Theory Is All You Need: AI, Human Cognition, and Causal Reasoning

We argue that AI’s data-based prediction is different from human theory-based causal logic and reasoning. We highlight problems with the decades-old analogy between computers and minds as input–output devices, using large language models as an example.

We introduce the idea of data–belief asymmetries to highlight the difference between AI and human cognition, using the example of heavier-than-air flight to illustrate our arguments. Theory-based causal reasoning provides a cognitive mechanism for humans to intervene in the world and to engage in directed experimentation to generate new data.

Teppo Felin, Matthias Holweg (2024) Theory Is All You Need: AI, Human Cognition, and Causal Reasoning. Strategy Science 9(4):346-371.

https://doi.org/10.1287/stsc.2024.0189

What’s up for the coming week?

I am happy to start new projects, both on Social AI in local communities and in the IoT community of practice. Next to that, the session we organise with Cities of Things at Dutch Design Week is approaching in two weeks (October 19). If you like to secure a spot at the workshop, **you can RSVP here!**

We also have the Call for Proposals still open for sessions at TH/NGS 2026. Check it here.

The event season is in full swing. I am not fully caught up yet, though. Check things like Robo Con, AI filmmaking. And there are numerous side events by local partners of WSAI. If that is your thing.