Poetry in Motion - My Generative AI eInk Display Clock
One of the first projects I ever did with generative AI (outside of just playing around with ChatGPT of course) was creating an eInk display clock that shows a generated poem. This was modeled after a popular kickstarter project, the Poem/1. It was my first interaction with the OpenAI API, eInk displays, and a raspberry pi pico w.
The goal of the project was to reproduce the Poem/1 project on kickstarter. That project featured a large eInk display that was connected to a web service that in turn generated poetry based on the current date and time. Because the clocks communicate with a centralized server, the only person who has to pay for the (quite expensive) OpenAI API calls is whoever is running the server.
For my project, I focused on getting a Raspberry Pi Pico W (the W stands for Wi-Fi, probably) working with an eInk display, both components being easily available on Amazon. We unfortunately don’t have much for electronic components where I live, so relying on online retailers is one of the few options available. I settled on an eInk display that sits nicely on the Pico pins, roughly the same size as the original board, and the whole thing runs off a micro-USB power supply.
This was my first time working with a microcontroller as well. I followed the guide at raspberrypi.com and got the MicroPython bios installed on the device. Once that was done I installed Thonny, the recommended IDE for playing around with MicroPython, and got to work.
The full code is available on my github repository, but we’ll break it down section-by-section here. I am by no means a python or microcontroller expert, but this code is likely enough for you to get started with your own projects with some modification.
import network
import time
#connect to wifi
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect("Network Name", "Password")
#init eink display
from driver import EPD_2in13_V4_Landscape
epd = EPD_2in13_V4_Landscape()
epd.Clear()
epd.fill(0xFF)
epd.text("Waiting to connect...", 5, 5, 0x00)
epd.display(epd.buffer)
#wait until connected
while not wlan.isconnected() and wlan.status() >= 0:
time.sleep(1)
Step one was setting up the wifi, which was easy enough to do from example scripts. But it was hard to tell if it was actually doing something… so I worked on printing to the eInk display next. This took quite a bit of figuring as there are lots of examples on how to connect to the eInk display through a regular Raspberry Pi that runs its own operating system, but not so many for a Pico that was running MicroPython. Luckily someone’s done the hard work and posted an open source repo at micropython-waveshare-epaper. Using the specific driver file there I was able to write some text to the screen indicating that we were waiting to connect to wifi, and then eventually connected.
import urequests
import ujson
import utime
#pull current time from jsontest.com
timeReq = urequests.get("http://date.jsontest.com")
time = timeReq.json()['milliseconds_since_epoch']
timeReq.close()
time = (time / 1000.0) - (60*60*6) #gmt -6
timeParts = utime.gmtime(int(time))
timeParts = (timeParts[0], timeParts[1], timeParts[2], 0, timeParts[3], timeParts[4], timeParts[5], 0)
print(timeParts)
#set internal clock
rtc = machine.RTC()
rtc.datetime(timeParts)
After that was done, I worked on figuring out the current date and time. The RPi Pico does have an onboard real time clock module, but there’s no battery backup, so every time you power it on you have to set the current date and time. I queried an online API available at date.jsontest.com and was able to parse the milliseconds_since_epoch from the response. It was about now that I realized that some of the objects and functions I was used to working with, particularly around date time manipulation, weren’t going to be available to me in MicroPython…
MicroPython is a subset of Python that is optimized to run on microcontroller hardware (like the raspberry pi pico w). As such, you can’t just port over existing Python code or bring in your favourite libraries; you either have to find an optimized alternative, or figure out another method. The Raspberry Pi Pico only has 2 MB of flash memory to work with, and that includes the MicroPython firmware, so you don’t really have a lot of room to work. Luckily a lot of common functionality is captured in libraries like utime, ujson, and urequests so you can usually figure something out.
while True:
now = localtime()
dt = "{:04d}/{:02d}/{:02d} {}:{:02d} {}".format(now[0], now[1], now[2],
(now[3] - 12 if now[3] > 12 else now[3]), now[4], ("PM" if now[3] >= 12 else "AM"))
We’ve got the time set, so time to make a clock. I start my loop by calling localtime(), which returns an array of the current date and time separated into pieces - each index corresponds to year, month, day, hour, minute, etc. I use this to build a formatted string that I display in the top left of the screen.
#if it's not working hours, just show a generic message
if(now[3] < 8 or now[3] >= 16):
epd.fill(0xff)
epd.text(dt, 5, 10, 0x00)
epd.text('You shouldn\'t be at your', 19, 51, 0x00)
epd.text('desk right now...', 57, 68, 0x00)
epd.init()
epd.display(epd.buffer)
epd.sleep()
sleep(60)
continue
Once I had the project up and running I quickly realized how expensive API calls to OpenAI were (I go through about $10 USD/month), so I quickly added a check to see if I’m actually at my desk before making an API call :) If the current hour is less than 8 or greater than 16 (4 PM), display a generic “you shouldn’t be working right now!” message.
data = ujson.dumps({
'model': 'gpt-4-turbo-preview',
'temperature': 1.4,
'messages': [
{ 'role': 'system', 'content': prompts[promptKey]['role'] },
{ 'role': 'user', 'content': prompts[promptKey]['prompt'].format(dt) }
]
})
chat = urequests.post("https://api.openai.com/v1/chat/completions", headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer {put your key here}'},
data = data)
response = chat.json()
if not 'choices' in response:
print(response)
continue
fullpoem = chat.json()['choices'][0]['message']['content']
chat.close()
Here I’m making an API call to OpenAI to ask them to generate a poem for me given the current time. I’m pulling the prompt from an array as I was experimenting with different types of content generation (i.e. ‘tell me a joke’, ‘tell me a historic fact about today’, ‘tell me about a famous person born on this day’), but ultimately none of the other prompts were as interesting as the poetry generation.
The poetry generation prompt is specifically:
{
'role': 'You are a poet, specializing in two-line poems.
You only return the poem in your responses.
The poem must always contain the current time.',
'prompt': 'The current time is {}'
}
where the current date & time would be injected into the prompt (yay retrieval-augmented generation!). You can see here I’m using the concept of a ‘system role’ as well, which is a powerful tool to help shape the content that the large language model generates.
#some formatting of the generated text
poem = ' / '.join([line.strip().rstrip(',').replace("’", "'").replace("—", "-") for line in fullpoem.splitlines() if line])
epd.fill(0xff)
epd.text(dt, 5, 10, 0x00)
#break generated text into lines that fit on display
lines = []
while(len(poem) > 32):
lineIndex = poem.rindex(' ', 0, 32)
lines.append(poem[0:lineIndex].strip())
poem = poem[lineIndex:]
lines.append(poem.strip())
After that is some formatting of the response. I found that the generated response had a few symbols that weren’t supported by the eInk driver’s system font (namely quotes and hyphens), and it returned the response broken over several lines. I replace those characters and join the broken up lines with a ` / ` to make it look like a fancy poem (and to optimize space since we only have so much to work with).
You’ll find that more often than not when working with generated text output you’ll need to apply some level of formatting. Here I’m only cleaning up the characters and spacing, but in other instances I’ve had to do things like parse out code blocks, look for specific start/end keywords, etc. It really depends on your use case. You have to treat generated text input as if it was free-form user input - not only is it unpredictable, but it can potentially include restricted keywords. It should be treated as unsafe text as necessary.
row = 1
#calculate center of poem area (not including datetime header)
textsize = len(lines) * 15
avail = 122 - textsize
offset = int(avail / 2)
#print each line
for line in lines:
rowavail = 250 - (len(line) * 8)
rowoffset = int(rowavail / 2)
epd.text(line, rowoffset, (offset + row * 15), 0x00)
row += 1
epd.init()
epd.display(epd.buffer)
Once we’ve got our poem all formatted, we print it on the screen. The eInk display driver uses a framebuffer to “virtually” draw what you want to an in-memory representation of the screen, and then once you’re done with that you can tell the driver to take that framebuffer and actually draw it on the display.
eInk is a unique display in that it does entire screen refreshes every time you update it (although some support partial refreshes, it’s not very common) - you’ve probably noticed this if you’ve used an eReader, how each new page has a full screen refresh. You don’t want ot flash the display with every line of text drawn, so instead we write it to an in-memory buffer (which is essentially a 2D array of which pixels to color in and which to blank out), and then send that entire array to the display driver to render.
epd.sleep()
now = localtime()
#wait until next minute
sleep(60 - now[5])
Finally we tell the eInk display driver that it can release its handle on the display since we won’t be doing anything else for awhile, and then figure out how long until the next minute, running a thread sleep until then.