Open AI, Algorithms and Art

Dalle-E Collection

I had forgotten I had signed up for an account with the DALL-E art site, which has gotten a fair share of notice for how it uses AI software to create art from written prompts. So when I saw an email yesterday, telling me my account was now active, I went in and played around. I used music themes for all of my prompts for the AI. The more specific the writing, the more interesting the image that the AI kicks out, I found.

I decided to create a “band” of musicians, with different settings and textual descriptions. It was an interesting experiment, and I used the “variations” tab quite a bit to see what the AI might generate in a second variation but for the most part, these come from the first round of algorithmic art by the platform.

I’ve included the text I used for the AI to generate the images.

DALL·E trumpet
DALL·E saxophone
DALL·E piano
DALL·E guitar
DALL·E drummer
DALL·E bass

You get a certain number of “credits” and then it costs some money to generate art.

Overall, I found the experience rather interesting, and yet I wondered how the AI was using my text descriptions to make itself “smarter” and was curious about what was going on underneath all of the code. There is a research paper available and the “about page” is full of positive elements of AI and the DALL-E site. It acknowledges the worries about AI, too, which I appreciated.

From the site:

Preventing Harmful Generations

We’ve limited the ability for DALL·E 2 to generate violent, hate, or adult images. By removing the most explicit content from the training data, we minimized DALL·E 2’s exposure to these concepts. We also used advanced techniques to prevent photorealistic generations of real individuals’ faces, including those of public figures.

Curbing Misuse

Our content policy does not allow users to generate violent, adult, or political content, among other categories. We won’t generate images if our filters identify text prompts and image uploads that may violate our policies. We also have automated and human monitoring systems to guard against misuse.

 

I am also curious about this part of the Mission Statement:

Our hope is that DALL·E 2 will empower people to express themselves creatively. DALL·E 2 also helps us understand how advanced AI systems see and understand our world, which is critical to our mission of creating AI that benefits humanity.

Let’s hope so, eh?

Peace (and Art),
Kevin

DALL·E music note

Nerdwriter: Dark Patterns

This one has been in my blog draft bin for some time. Worth re-visiting for understanding better how companies try to manipulate us (users) to gather more information and to keep us inside their tents.

I support Nerdwriter through Patreon.

Peace (breaking out),
Kevin

Poetry: Intersections of Words, Art, Music and Technology

Algorithmic Artists and the Solo Saxophonist

I’ve been sharing out some of my morning poems, where I have been exploring the intersections of art and music and writing with technology. The above poem was inspired by an AI site — Dream — that creates art from keywords (here, my words were Saxophone Nights). I used the image, along with explorations this week with Hour of Code and programming, to spark the idea for the poem.

This morning, after a helpful remembering by Wendy T. yesterday, I used JazzKeys to craft a poem, with jazz piano as a soundtrack for each time my fingers hit the computer keyboard in the spur-of-moment writing. I just let the words flow as I listened to the piano. (I am listening now as I write this, too)

Listen: https://jazzkeys.plan8.co/?msg=-MqOfDSE_Pl2R2bBSNTx 

I also created a blended visual of the same poem with a piano player, using a screenshot of the JazzKeys poem and a Creative Commons image, then merged with Lunapic. I like the ghost notes aspect of the result, as the words are fading (and if you listen to the JazzKeys as you read, the experience is even better, I think).

Ghost Notes

Peace (listening),
Kevin

Sixth Flight (visualization remix)

In my continuing explorations of word and sound, I saw that my friend, Ron, had used a site called Specterr to create a visualization for a Daily Create in DS106.

I decided to check it out and then realized I could further remix the audio of my Sixth Bird in Flight poem from last week (see post about that project), but I quickly knew that the raw MIDI audio track (which was a music file that an AI site created out of the text of a poem) could use something more — more layers, more colors, more thickening.

I added a few layers of instrumentation and a beat underneath the music file (that was a conversion of the text of my poem), and I liked how it came out when my remixing was done. Then I uploaded the audio file into Specterr (in its free account, which is why there is a watermark on it), and tinkered with some animation and color and more.

I like seeing the audio visualization in sync with the beat.

Peace (kicking it in),
Kevin

Turning Text into Music (A Small AI Experiment)

careful now
careful now flickr photo by fibreman shared into the public domain using Creative Commons Public Domain Dedication (CC0)

I was curious the other week about whether I might find any online AI-powered sites that take text and turn those words into music. Honestly, I didn’t think my search would be all that successful. But it was. To a point.

Here are three (free) online sites powered by algorithms that I found and tinkered with. I am going to use the paragraph I just wrote as the intro I just wrote to this post as the text that I want each site to turn into music (See words above). Each of the sites will use the same exact text.

First, there is Typatone, which is one my favorites here.  You can either type the letters/words, and hear the song as you write (which is pretty cool but if you are like me, I make a ton of mistake as I type and so that feature is less useful than it would seem). Or you can input your text and let the site do its thing.

One annoying quirk is that once it starts playing, there’s no pause button. Sorry. I think watching it and listening to it is the best, though, so just an audio file to listen to would feel rather empty as an experience.

Here is my test.

Another is Langorhythm 2.0 (which is explained in a neat TED Talk that assigns notes to letters). I liked this site for its simplicity but soon found that the timbre and instrumentation never changes from the original conversion — it is the same tones for every letter, which makes sense if you know how the algorithm was created. But it makes it sort of … a sameness when doing multiple projects.

That said, the site kicks out a MIDI audio file, which is quite useful for anyone who has a digital music workstation. I use Soundtrap, for example, and so it was easy to change the instrumentation from the generated Rhodes piano into something a little more … interesting.

Listen to original and then my remix (changing the sound/instruments)

The third, and both most robust and clear strangest of the sites, is Melobytes (I worked with the free Pro model — I assume it’s free for a limited time?). Here, you have many more setting options (too many, perhaps), although I was never sure how the piece would sound when I was done with it. And mostly, I found the site interesting in an analytical way but too randomized and jarring to be easy listening, no matter what I did with the settings (and you have a limit on free access to how many times you can reset the conversion). The videos (with AI chosen images) were weird every single time and the vocal sounds, even weirder.

What I did find fascinating, though,  with Melobytes was that the site creates a piece of musical manuscript for the inputted text that forms the basis of the audio file it generates. Here’s my same blog intro, written now as music that I could play. I don’t know the critera for how it determined length or tone of notes, etc. (And I think there may have been a second page with the word “point” on it that I missed.)

What does this all mean? I don’t rightly know. I have only a vague notion of how the sites took my words and kicked out sound. As a writer who loves sound and music, and is both interested and skeptical of the age of AI, I find these experiments to be helpful in understanding both how far computers have come (caveat: free online availability) and how far they have to go.

Peace (sounds like),
Kevin

Digital Poetry Process Notes: Launching Birds of Flight

Last week, I released six “birds in flight” poems, one per day. Here, I’d like to provide some context notes and process decisions, as well as tech tools,  for each poem, as both a way to share my digital compositional practices, to reflect on what worked and didn’t and why, and to archive my notes, for my future self (hello, me).

My poetry collection began not with writing but with reading a poem and sharing it with friends. In an edition of Orion, a nature-writing journal, there was a lovely card stock pullout of a poem called “Poetry” by Chun Yu (one side was English, and the other side, Chinese). I can’t find it online to share here, and I don’t want to infringe on copyright by sharing myself. But the poem was lovely, with a theme of poetry.

I did share it with my poetry friends in the new closed NWPStudio Space, however, and my collaborator and colleague and poetry ping-pong companion, Terry Elliott, paid attention to and noticed the architecture of the poem. Terry then pulled out some guiding prompts that could become a flexible template for writing a poem, inspired by Chun Yu’s “Poetry.” It was from Terry’s excavation of ideas that I wrote a small poem each day, for six days, using the opening lines from Chun Yu’s poem of birds (see above) as my theme.

Here, then, are my six poems — my six Birds in Flight, if you will — and a reflection on how I created the digital versions of them and the decisions that I made to do so.

First Bird in Flight

For poems with small amounts of words, like these, a site like Lumen5 is perfectly situated. It is a digital storytelling site that allows many choices for image and video, with text, and music, and even the opportunity for voice-over (which I decided not to do here, as my experiment with it seemed to take away from the contemplative nature of the visual poem.) The most important decision here for me became the soundtrack, and I grappled with how the music would inform the words and image. In the end, I found what I still think is a perfect sound companion to the poem — it gives it just enough calm, and includes the sounds of birds.

Second Bird in Flight

I knew I would be diving into digital composition with many of these small poems so I wanted to hand-write out a poem, as sort of a counter-measure to complete immersion with digital. Of course, I did it via an app (Sketchpad), and then thought about a reflecting pool or image of the poem, and remembered an effect in LunaPic that could do that. It worked nicely, I think, given the hand-written text a little more depth and wrinkle, or maybe, ripples. That I chose a piece of paper theme for the writing makes it even more interesting in the reflection, I think.

Third Bird in Flight

This poem used an app I tap into quite a bit for animating words. It is called TypiVideo and it has some strange quirks (you can’t control what words on a single screen at any time after you input your entire text, so there are often odd endings of phrasings). But there are neat options for font and color, and I like how the text flows forward, and have come to appreciate the unexpected breaks. After creating it, I felt as if it were missing something, so I created a music track with thoughts of flight and layered it in, giving the words some ambience.

Fourth Bird in Flight

This poem used a text animation app on my iPad called PLAYS that I come back to now and then. It has a solid collection of animated options, some of which are too busy for any use by a poet, though. I found one that had the text moving off the screen like a bird in flight (similar to the next poem’s construction). But when I had built it and exported it, it seemed like it still needed something else. I pulled the animated image into LunaPic (always a useful image editing site) and found an effect that turned the piece yellow and faded at the edges, like a corn husk (sort of) that connected to a phrase in the poem. I like that when the words leave, there is an after-effect of a splotch of light yellow, as if the poem has left a mark.

Fifth Bird in Flight

I knew I wanted this poem to be its own poem but also to reveal a second poem after the words “flew away” into the sky. I used Keynote to do this, and it took some time, as different words and lines had to be their own text boxes that could be animated or remain stationary as the poem flew off like birds in flight. (This could have been done easily enough in Google Slides or Powerpoint, too). I like that the poem I left behind or that was uncovered is more positive and optimistic than the poem pieces that depart. Exporting as a GIF allows for the poem to reset itself each time.

Sixth Bird in Flight

This last poem was the most complex composition of the collection. I had been curious about whether I could turn the words of this poem into music. Of course, I could have sat down with my guitar, but I wanted to push the concept of AI, so I searched around and found two sites: Langorythm (which converts words into midi-file music – see this) and Melobytes (which converts text into what I can only describe as an odd piece of music, with even stranger video, and also, interestingly, a piece of music manuscript). I had been tinkering with both when I realized I could merge the output from both sites, using the melodic music that Langorythm created from the text of the poem beneath the video and manuscript created from my poem by Melobytes, and then, realizing this composition needed some stability to center the actual poem, I added my voice overlay to video. At one point, I had an entire earlier version of this project that I did all the work on, as finished project, and then I could not shake the sense that the “feel” of the Melobytes output was all wrong. So I started over from the beginning, and began construction again. The result is something interesting, if unusual.

I hope this both helps me to remember what I did, but also inspired YOU to tinker and play with digital compositions, to see how we might use technology to further put poems into motion while trying to deepen the composition’s impact on a reader, viewer, listener.

Peace (on wings of words),
Kevin

Book Review: You Look Like a Thing and I Love You (How Artificial Intelligence Works and Why It’s Making the World a Weirder Place)

It’s not easy to wrap the head around the role of algorithms in our social networks and our other online spaces but there they are — always on and always working and maybe causing trouble with unanticipated consequences. Just look at how algorithms push bias and racism, and lead us towards uncharted waters.

Janelle Shane, in her book You Look Like A Thing And I Love You does a fantastic job of explaining the ins and outs of algorithm design and workings, while keeping the mood light and entertaining with humorous stories and simple doodles — but never pure fluff. You’ll learn about mathematical modeling and algorithm coding and be reminded, again and again, how dumb algorithms really are.

It’s the human programmers that are the problem. That, and the ways that algorithms obscure what they are doing deep inside code, so that even as they are learning from past experiences and making adaptations to become more efficient in their job, it is not always clear how they are doing the work they are doing, once launched into their tasks. Shane dispels notions of algorithms taking over the world at any time in the near future, but she does warn that we need to have more openness and clarity about how algorithms work, so we can fix them when they go wrong.

Shane knows her topic well, and knows how to explain it to a non-programming audience. I found her book both entertaining and informative.

Peace (coded for now),
Kevin