Maria

Power up your Raspberry Pi. SSH into your Pi from a computer on the same network.

In your terminal, navigate to whichever directory you want to run Maria from. It doesn't really matter. Clone my Github repository into your folder.

That might look something like cd Documents && git clone https://github.com/jessp/holo-assistant.git.

Alternatively, you can download my code from here.

I wanted Maria to be as easy and affordable to assemble as possible. That said, you need some things to get started. I've linked to some of the specific items I used (no affiliate links), but please consider alternatives that might be easier to source for you.

Materials

• Raspberry Pi 4B running a recent Raspberry Pi OS
• Micro SD card and charger, for your Raspberry Pi
• The cheapest keyboard you can find, for set-up
• The cheapest mouse you can find, for set-up
• A 3.2", 800x400 screen, ideally one that connects over HDMI
• Small USB speaker
• Miniature USB microphone
• Acetate sheets
• 3D-printed case (optional), composed of:
• Base for Raspberry Pi and peripherals (PLA FDM)
• 3x standard walls for the base (PA12 Nylon SLS)
• 1x wall with USB hole for the base (PA12 Nylon SLS)
• Lid for base (PLA FDM)
• Crown structure for surrounding cone (PA12 Nylon SLS)
• Cap for crown structure (PLA FDM)

I note that the case is optional because, honestly, it's a bit pricy to 3D print, and my modelling skills aren't great. You can absolutely use those files, but I also encourage you to be creative with whatever materials you have around.

The case in the videos above is an older model, but you can see the updated version in this Wiki post.

Maria is powered by a few different APIs. You'll need to sign up for accounts in order to interact with her.

APIs

• Signing up for WeatherAPI.com is pretty straight-forward.
• Sign up for a Spotify developer account. Create an app. Under the app's settings, enter http://localhost:9876 under Redirect URIs.
• Sign up for Words API via RapidAPI.
• Sign up for Google Cloud's Text-to-Speech service.
• I find that Google Cloud's services a bit tricky to navigate, so I recommend following the steps here under the headings "Enable the Cloud Text-to-Speech API" and "Set Up Service Account."
• Place the json file you download in holo-assistant/serverAudio/googleCredentials.

Once you have all your keys, create a new file called settings.config in holo-assistant/serverAudio. Copy the structure of the file from settings.config.example, but fill in the keys you acquired by signing up for the API services.

Still SSH-ing into your Pi, create a virtual environment at the top-level of your holo-assistant directory named "holovenv."" To do this, run the command python3 -m venv holovenv.

Activate the environment by running source holovenv/bin/activate. While in holovenv, do the following:

• Install Python libraries by running pip install -r serverAudio/requirements.txt
• Run sudo apt-get install libatlas-base-dev and sudo apt install python3-pyaudio to prevent some Raspberry Pi-specific issues.

Exit your virtual environment.

Via SSH, navigate outside of your holo-assistant directory to where you want to download Raylib to. Raylib is the 3D library used to animate Maria.

Download Raylib using git clone https://github.com/raysan5/raylib. Navigate into your newly download Raylib folder.

Raylib has a pretty great Wiki on installing Raylib on Raspberry Pi.

At the risk of mostly reiterating the steps described in the Wiki entry, I did the following:

• Run sudo apt-get install --no-install-recommends raspberrypi-ui-mods lxterminal gvfs
• Run sudo apt-get install libx11-dev libxcursor-dev libxinerama-dev libxrandr-dev libxi-dev libasound2-dev mesa-common-dev libgl1-mesa-dev
• Navigate into raylib/src and then run sudo make install PLATFORM=PLATFORM_DESKTOP GRAPHICS=GRAPHICS_API_OPENGL_21

If you end up faffing around with nothing working, try enabling x11 forwarding on your Pi per this article. I never quite confirmed if it was necessary for Raylib.

Plug your crappy keyboard and mouse into your Raspberry Pi. If you haven't already, connect your screen as well.

Use Raspberry Pi's settings to rotate your screen orientation and select your speakers as your audio output. I promise, it's so much easier using the GUI than trying to do this over SSH.

Open the terminal on your Raspberry Pi. Navigate into holo-assistant. Run cd serverAudio && ./../holovenv/bin/python runmefirst.py.

The main thing runmefirst.py does is trigger the Spotify authentication process, which must be done within the browser of the device you are using it on. It also includes some calls to attempt to trigger a geography library we use, geograpy3, to download some essential place-name data we need before we can use it. But that call is pretty hit and miss, and doesn't impact our ability to run Maria.

Unplug your keyboard and mouse.

Maria needs a place to live. All you need is cone made out of an acetate sheet of a paper, but you can also 3D print a special case if you so choose. The video below will walk you through how to assemble Maria's home, and the steps are written out after. Some details of the case have been updated since the video was made, which are documented here.

Instructions

1. Draw two circles on a sheet of acetate with the same centre point. The smaller circle has a radius of 1.5cm, while the larger circle has a radius of 10cm.
2. Draw a 90° angle extending from the centre of the circles.
3. Draw jagged rectangular "teeth" extending from one line making up the angle.
4. Cut out your shape. This will form your cone.
5. Tape your cone together so that your jagged teeth overlao with the other edge.
6. If you haven't already, plug in your screen to your Raspberry Pi.
7. Plug in your microphone to your Raspberry Pi.
8. Glue together the walls with the notches facing inward.
9. Snap fit the walls onto the base.
10. Plug in your speaker to your Raspberry Pi. Place the speaker in the four corners that extend up from the base of the Pi case.
11. Position your Raspberry Pi on the raised platform. The bottom of the screen should face towards the speaker.
12. Plug your power cord into the Raspberry Pi through the hole in one wall, then put the lid of the base on. The screen should line up with the rectangular cut-away of the lid.
13. Position your cone on the lid of the case. The bottom third of the narrow end of the cone should overlap with the screen.
14. Place the cage over your cone. The gap in the cage should face the side of the case the power cable extends from.
15. Finally, place the cap over the cage.

Open two terminal tabs that both are SSH-ing into your Raspberry Pi. In both tabs, navigate into holo-assistant.

In one, run cd serverAudio && ./../holovenv/bin/python main.py.

In the other, run export DISPLAY=:0.0 and then cd clientCharacter && cc holoAssistant.c character.c stopWatch.c -lraylib -lGL -lm -lpthread -ldl -lrt -lX11 -o HoloAssistant && ./HoloAssistant.

Maria loves to help her friends. She's also a buggy work-in-progress, so there are some limitations as to what she can do. To get Maria's attention, say her name. Then ask her for help with any of the things below.

Music

• Say "play" followed by the name of a song to play a song on Spotify
• Use the phrase "as by" to specify a performer. For example, you might say "play 'The Man Who Sold the World' as by Nirvana"
• Say "stop", "pause", or "cancel" the "song" or the "music" to stop playback
• This only works on devices that you've very recently played Spotify music on
• This doesn't work on Sonos devices unless you're using "AirPlay" or something similar
• I'm using Vosk's English language model, so this works best with English song names

Weather

• If you just ask for the weather (e.g., "What's the weather?"), Maria will use your IP address to determine your location via WeatherAPI.com's API, and will say the weather for the rest of the day
• Say "tomorrow" to specify's tomorrow's weather rather than today's
• If you say "in" followed by a city name (e.g., "What's the weather in Paris?"), Maria will attempt to get weather in that location

Timer

• Maria can set timers while you go about other things
• Say "stop" or "cancel" the "timer" to stop the timer
• You can set the time in seconds, minutes, and hours
• You cannot set a timer for longer than six hours

Dictionary

• Maria can define words when given prompts like, "define", "what's the meaning of", or "what's the definition of"