I had forgotten I had signed up for an account with the DALL-E art site, which has gotten a fair share of notice for how it uses AI software to create art from written prompts. So when I saw an email yesterday, telling me my account was now active, I went in and played around. I used music themes for all of my prompts for the AI. The more specific the writing, the more interesting the image that the AI kicks out, I found.
I decided to create a “band” of musicians, with different settings and textual descriptions. It was an interesting experiment, and I used the “variations” tab quite a bit to see what the AI might generate in a second variation but for the most part, these come from the first round of algorithmic art by the platform.
I’ve included the text I used for the AI to generate the images.
You get a certain number of “credits” and then it costs some money to generate art.
Overall, I found the experience rather interesting, and yet I wondered how the AI was using my text descriptions to make itself “smarter” and was curious about what was going on underneath all of the code. There is a research paper available and the “about page” is full of positive elements of AI and the DALL-E site. It acknowledges the worries about AI, too, which I appreciated.
We’ve limited the ability for DALL·E 2 to generate violent, hate, or adult images. By removing the most explicit content from the training data, we minimized DALL·E 2’s exposure to these concepts. We also used advanced techniques to prevent photorealistic generations of real individuals’ faces, including those of public figures.
Our content policy does not allow users to generate violent, adult, or political content, among other categories. We won’t generate images if our filters identify text prompts and image uploads that may violate our policies. We also have automated and human monitoring systems to guard against misuse.
I am also curious about this part of the Mission Statement:
Our hope is that DALL·E 2 will empower people to express themselves creatively. DALL·E 2 also helps us understand how advanced AI systems see and understand our world, which is critical to our mission of creating AI that benefits humanity.
I had been revisiting a crowd-sourced resource of technology tools and platforms recently, and noticed that many of the sites were dead and gone. So I invited a few friends in CLMOOC and beyond to help update the list. We curated it with general themes to help give some semblance of order to it. We hope it’s helpful.
This one has been in my blog draft bin for some time. Worth re-visiting for understanding better how companies try to manipulate us (users) to gather more information and to keep us inside their tents.
I’ve been sharing out some of my morning poems, where I have been exploring the intersections of art and music and writing with technology. The above poem was inspired by an AI site — Dream — that creates art from keywords (here, my words were Saxophone Nights). I used the image, along with explorations this week with Hour of Code and programming, to spark the idea for the poem.
This morning, after a helpful remembering by Wendy T. yesterday, I used JazzKeys to craft a poem, with jazz piano as a soundtrack for each time my fingers hit the computer keyboard in the spur-of-moment writing. I just let the words flow as I listened to the piano. (I am listening now as I write this, too)
I also created a blended visual of the same poem with a piano player, using a screenshot of the JazzKeys poem and a Creative Commons image, then merged with Lunapic. I like the ghost notes aspect of the result, as the words are fading (and if you listen to the JazzKeys as you read, the experience is even better, I think).
In my continuing explorations of word and sound, I saw that my friend, Ron, had used a site called Specterr to create a visualization for a Daily Create in DS106.
I decided to check it out and then realized I could further remix the audio of my Sixth Bird in Flight poem from last week (see post about that project), but I quickly knew that the raw MIDI audio track (which was a music file that an AI site created out of the text of a poem) could use something more — more layers, more colors, more thickening.
I added a few layers of instrumentation and a beat underneath the music file (that was a conversion of the text of my poem), and I liked how it came out when my remixing was done. Then I uploaded the audio file into Specterr (in its free account, which is why there is a watermark on it), and tinkered with some animation and color and more.
I like seeing the audio visualization in sync with the beat.
I was curious the other week about whether I might find any online AI-powered sites that take text and turn those words into music. Honestly, I didn’t think my search would be all that successful. But it was. To a point.
Here are three (free) online sites powered by algorithms that I found and tinkered with. I am going to use the paragraph I just wrote as the intro I just wrote to this post as the text that I want each site to turn into music (See words above). Each of the sites will use the same exact text.
First, there is Typatone, which is one my favorites here. You can either type the letters/words, and hear the song as you write (which is pretty cool but if you are like me, I make a ton of mistake as I type and so that feature is less useful than it would seem). Or you can input your text and let the site do its thing.
One annoying quirk is that once it starts playing, there’s no pause button. Sorry. I think watching it and listening to it is the best, though, so just an audio file to listen to would feel rather empty as an experience.
Another is Langorhythm 2.0 (which is explained in a neat TED Talk that assigns notes to letters). I liked this site for its simplicity but soon found that the timbre and instrumentation never changes from the original conversion — it is the same tones for every letter, which makes sense if you know how the algorithm was created. But it makes it sort of … a sameness when doing multiple projects.
That said, the site kicks out a MIDI audio file, which is quite useful for anyone who has a digital music workstation. I use Soundtrap, for example, and so it was easy to change the instrumentation from the generated Rhodes piano into something a little more … interesting.
The third, and both most robust and clear strangest of the sites, is Melobytes (I worked with the free Pro model — I assume it’s free for a limited time?). Here, you have many more setting options (too many, perhaps), although I was never sure how the piece would sound when I was done with it. And mostly, I found the site interesting in an analytical way but too randomized and jarring to be easy listening, no matter what I did with the settings (and you have a limit on free access to how many times you can reset the conversion). The videos (with AI chosen images) were weird every single time and the vocal sounds, even weirder.
What I did find fascinating, though, with Melobytes was that the site creates a piece of musical manuscript for the inputted text that forms the basis of the audio file it generates. Here’s my same blog intro, written now as music that I could play. I don’t know the critera for how it determined length or tone of notes, etc. (And I think there may have been a second page with the word “point” on it that I missed.)
What does this all mean? I don’t rightly know. I have only a vague notion of how the sites took my words and kicked out sound. As a writer who loves sound and music, and is both interested and skeptical of the age of AI, I find these experiments to be helpful in understanding both how far computers have come (caveat: free online availability) and how far they have to go.
Last week, I released six “birds in flight” poems, one per day. Here, I’d like to provide some context notes and process decisions, as well as tech tools, for each poem, as both a way to share my digital compositional practices, to reflect on what worked and didn’t and why, and to archive my notes, for my future self (hello, me).
My poetry collection began not with writing but with reading a poem and sharing it with friends. In an edition of Orion, a nature-writing journal, there was a lovely card stock pullout of a poem called “Poetry” by Chun Yu (one side was English, and the other side, Chinese). I can’t find it online to share here, and I don’t want to infringe on copyright by sharing myself. But the poem was lovely, with a theme of poetry.
I did share it with my poetry friends in the new closed NWPStudio Space, however, and my collaborator and colleague and poetry ping-pong companion, Terry Elliott, paid attention to and noticed the architecture of the poem. Terry then pulled out some guiding prompts that could become a flexible template for writing a poem, inspired by Chun Yu’s “Poetry.” It was from Terry’s excavation of ideas that I wrote a small poem each day, for six days, using the opening lines from Chun Yu’s poem of birds (see above) as my theme.
Here, then, are my six poems — my six Birds in Flight, if you will — and a reflection on how I created the digital versions of them and the decisions that I made to do so.
First Bird in Flight
For poems with small amounts of words, like these, a site like Lumen5 is perfectly situated. It is a digital storytelling site that allows many choices for image and video, with text, and music, and even the opportunity for voice-over (which I decided not to do here, as my experiment with it seemed to take away from the contemplative nature of the visual poem.) The most important decision here for me became the soundtrack, and I grappled with how the music would inform the words and image. In the end, I found what I still think is a perfect sound companion to the poem — it gives it just enough calm, and includes the sounds of birds.
Second Bird in Flight
I knew I would be diving into digital composition with many of these small poems so I wanted to hand-write out a poem, as sort of a counter-measure to complete immersion with digital. Of course, I did it via an app (Sketchpad), and then thought about a reflecting pool or image of the poem, and remembered an effect in LunaPic that could do that. It worked nicely, I think, given the hand-written text a little more depth and wrinkle, or maybe, ripples. That I chose a piece of paper theme for the writing makes it even more interesting in the reflection, I think.
Third Bird in Flight
This poem used an app I tap into quite a bit for animating words. It is called TypiVideo and it has some strange quirks (you can’t control what words on a single screen at any time after you input your entire text, so there are often odd endings of phrasings). But there are neat options for font and color, and I like how the text flows forward, and have come to appreciate the unexpected breaks. After creating it, I felt as if it were missing something, so I created a music track with thoughts of flight and layered it in, giving the words some ambience.
Fourth Bird in Flight
This poem used a text animation app on my iPad called PLAYS that I come back to now and then. It has a solid collection of animated options, some of which are too busy for any use by a poet, though. I found one that had the text moving off the screen like a bird in flight (similar to the next poem’s construction). But when I had built it and exported it, it seemed like it still needed something else. I pulled the animated image into LunaPic (always a useful image editing site) and found an effect that turned the piece yellow and faded at the edges, like a corn husk (sort of) that connected to a phrase in the poem. I like that when the words leave, there is an after-effect of a splotch of light yellow, as if the poem has left a mark.
Fifth Bird in Flight
I knew I wanted this poem to be its own poem but also to reveal a second poem after the words “flew away” into the sky. I used Keynote to do this, and it took some time, as different words and lines had to be their own text boxes that could be animated or remain stationary as the poem flew off like birds in flight. (This could have been done easily enough in Google Slides or Powerpoint, too). I like that the poem I left behind or that was uncovered is more positive and optimistic than the poem pieces that depart. Exporting as a GIF allows for the poem to reset itself each time.
Sixth Bird in Flight
This last poem was the most complex composition of the collection. I had been curious about whether I could turn the words of this poem into music. Of course, I could have sat down with my guitar, but I wanted to push the concept of AI, so I searched around and found two sites: Langorythm (which converts words into midi-file music – see this) and Melobytes (which converts text into what I can only describe as an odd piece of music, with even stranger video, and also, interestingly, a piece of music manuscript). I had been tinkering with both when I realized I could merge the output from both sites, using the melodic music that Langorythm created from the text of the poem beneath the video and manuscript created from my poem by Melobytes, and then, realizing this composition needed some stability to center the actual poem, I added my voice overlay to video. At one point, I had an entire earlier version of this project that I did all the work on, as finished project, and then I could not shake the sense that the “feel” of the Melobytes output was all wrong. So I started over from the beginning, and began construction again. The result is something interesting, if unusual.
I hope this both helps me to remember what I did, but also inspired YOU to tinker and play with digital compositions, to see how we might use technology to further put poems into motion while trying to deepen the composition’s impact on a reader, viewer, listener.
It’s not easy to wrap the head around the role of algorithms in our social networks and our other online spaces but there they are — always on and always working and maybe causing trouble with unanticipated consequences. Just look at how algorithms push bias and racism, and lead us towards uncharted waters.
Janelle Shane, in her book You Look Like A Thing And I Love Youdoes a fantastic job of explaining the ins and outs of algorithm design and workings, while keeping the mood light and entertaining with humorous stories and simple doodles — but never pure fluff. You’ll learn about mathematical modeling and algorithm coding and be reminded, again and again, how dumb algorithms really are.
It’s the human programmers that are the problem. That, and the ways that algorithms obscure what they are doing deep inside code, so that even as they are learning from past experiences and making adaptations to become more efficient in their job, it is not always clear how they are doing the work they are doing, once launched into their tasks. Shane dispels notions of algorithms taking over the world at any time in the near future, but she does warn that we need to have more openness and clarity about how algorithms work, so we can fix them when they go wrong.
Shane knows her topic well, and knows how to explain it to a non-programming audience. I found her book both entertaining and informative.
I was asked to write a piece over at NWP Write Now about the sudden rush to technology that has engulfed us all in the Pandemic, with a reminder that it is the teaching and teacher and pedagogy that is always more important than the app, site or platform. I found it helpful in the writing of the piece to remember my own advice.
And plenty of National Writing Project colleagues and I have been engaged in Twitter discussions about the viability of HyperDocs, as well as the limitations. It is important to note, as the authors do repeatedly, that HyperDocs are not just some amped up worksheet to be given remotely to students. (See Deanna Mascale’s latest post on Hyperdocs for her university instruction) I also know there are criticism of this kind of approach, as being too prescriptive or narrowing in scope for learners.
The three authors of The Hyperdoc Handbook are experienced teachers and instructional coaches and technology advocates, and I appreciated the approach of screenshots and examples and the way they talk through the pedagogical rationale for Hyperdocs as a way to engage all learners in a guided yet independent inquiry process. They explore pedagogy and tap into the ways that well-designed Hyperdocs can extend the idea of Zones of Proximal Development, through layered choices and skills and expectations.
You don’t need buy this book to learn about Hyperdocs (I am one of those own-a-book people and I like to support teachers) and their website has plenty of examples and templates and more that you can examine and borrow, and hack, as the authors tell you in the book. A blog post at the site even provides some useful thinking on remote teaching with Hyperdocs.
This week, in fact, I am going to use a HyperDoc with teachers as part of a professional development session on Project-Based Learning, in which teachers explore a theme for a short/tiny public service announcement (an idea borrowed from AJ Jacobs).
I’m deep into the design stage of curriculum for the start of our school year (which begins remote and then becomes hybrid, with independent learning days for students in the weekly schedule). I see some possibilities here for my students, although it is important to acknowledge that Hyperdocs as nothing new, really, but more of a way to organize resources for student inquiry and exploration. Webquests, websites, blog posts, etc, all are in the same family. The book is helpful in its range of examples, visuals and testimonials from other educators.
As mentioned, a Hyperdoc (which does NOT have to be a Google Doc or product) is definitely more than a glorified worksheet. It’s more like an anchor or docking point, leading students to other activities and resources. That’s important to remember.