Controlling Flappy Bird with my brain
Imagine playing Flappy Bird, not with your fingers, but simply by using your thoughts!
That sounds like something out of the future!
It’s not just a wild idea — it’s actually possible!
So, picture this: Flappy Bird, the classic game we all played at one point in our lives, but with a twist. Instead of tapping your screen like crazy to keep that bird alive, I figured out a way to control it using my brainwaves.
Basically, when blinking there's a change in the brainwaves of my brain activity which I took to my advantage and coded, so that every time I blinked, Flappy Bird would jump!
How do you measure the brainwaves in the first place though?
So, first of all, I measured the brainwaves by using the Muse Headband 2 which is an affordable EEG (Electroencephalogram) headset for beginners.
The Muse headband measures my brain activity by using different kinds of sensors such as:
- Electrooculography (EOG) Sensors: EOG sensors are integrated into the Muse Headband 2 to detect eye movements and blinks. These sensors distinguish between eye-related movements and genuine brainwave signals!
- Accelerometer Sensors: The Muse Headband 2 also has accelerometer sensors that measure head movements and position. These sensors measure the posture and eliminate motion-related parts from the EEG signals.
- PPG (Photoplethysmography) Sensors: PPG sensors measure heart rate and blood flow. By monitoring these physiological parameters the Headband can give additional information about the cognitive state and stress levels.
- Temperature Sensor: The headband also utilizes a temperature sensor to monitor the skin temperature. Changes in skin temperature can indicate variations in arousal or emotional states!
The main sensors on the Muse Headband employ EEG (electroencephalography) technology:
Electroencephalography is a non-invasive neuroimaging technique that measures and records electrical activity in the brain using sensors placed on the scalp.
The Muse Headband 2 uses dry EEG electrodes which don’t need conductive gel or paste that is typically used in traditional EEG setups and simplifies the process. The Muse Headband 2 consists of four EEG sensors strategically positioned on the forehead and behind the ears to capture electrical signals originating from different brain regions, and are made out of silver/silver chloride or carbon. They have good electrical conductivity and biocompatibility.
Each EEG sensor on the Muse Headband 2 has multiple components including a conductive material an electrode and an amplifier. The conductive material establishes a low-impedance connection between the electrode and the scalp which is important for efficient signal transmission. If you don’t know what a low-impedance connection is, it’s basically like having a clear path for electrical signals to travel. It means there’s minimal resistance, which is important for accurate measurements in e.g. EEG sensors. This helps to minimize noise interference from outside.
The electrodes measure the brain’s electrical activity through four channels: TP9, TP10, AF7, and AF8 on the mind monitor app I additionally downloaded. These sensors capture raw EEG signals, measured in microvolts (μV), and detect the potential differences (voltage) between the active electrodes and a reference electrode (Fpz). The headband’s frontal and temporal electrodes pick up electrical signals produced by the brain’s neurons and serve as the interface between the brain’s electrical activity and the measurement system.
To amplify the weak electrical signals picked up by the electrodes the Headband has an integrated amplifier. The amplifier boosts the signal strength while minimizing noise and interference.
But WHY are the brainwaves different when I’m blinking and HOW do they change?
There are 5 different types of brainwaves:
And what happens is, that every time you blink, there is an increase in delta-band amplitude, which reflects a specific pattern of brainwave activity!
Delta waves are the slowest kind of brainwaves, with a frequency range of up to 4 Hz, and they tend to be the highest in amplitude. These waves are typically associated with deep, dreamless sleep and are also frequently seen in infants. An increase in delta-band amplitude indicates a state of reduced brain activity and is linked to deep relaxation and unconsciousness. It is also connected to conditions such as coma, brain death, and some mental disorders!
Recent studies have found that the increase in delta-band amplitude, is related to the activation of a brain region called the precuneus.
This region is involved in continuously gathering information from the surrounding environment and is linked to awareness. When consciousness is low or absent, the activity in the precuneus is reduced or completely missing. Therefore, the increase in delta-band amplitude during blinking is connected to the brain’s process of monitoring the environment and also maintaining awareness.
The increase in delta-band amplitude also additionally reflects a decrease in cortical activity and is often observed during states of reduced awareness and low-level arousal.
This activation and deactivation of brain networks during blinking suggest that eyeblinks are actively involved in attentional disengagement during brain activity by momentarily activating the default-mode network while deactivating the dorsal attention network.
How do you code the game, to be able to control the Flappy Bird by blinking?
This part of the code imports three essential libraries: pygame, sys, and random.
- pygame is a library for creating games in Python
- sys provides access to some variables and functions used by the Python interpreter
- random is used for generating random numbers
This line initializes the Pygame library. It needs to be called before using any Pygame functions.
This section defines several constants used in the game.
- WIDTH and HEIGHT set the dimensions of the game window
- FPS determines the frame rate (how many frames per second the game runs)
- GRAVITY is the acceleration due to gravity affecting the bird
- BIRD_JUMP is the velocity applied to the bird when it jumps
- PIPE_VELOCITY is the speed at which pipes move from right to left
The WHITE constant is assigned an RGB color value representing white.
These variables are used to track the state of the game.
- bird_y represents the vertical position of the bird
- bird_velocity represents the vertical velocity of the bird
- pipes is a list that will hold the positions of pipes in the game
This code creates the game window using Pygame. The set_mode function takes the window dimensions as an argument, and set_caption sets the title of the window.
Images for the bird and pipes are loaded into the game. get_rect ( ) is called to get the rectangular area for each image, which will be used for positioning and collision detection.
- This code defines a function that is reset_game responsible for resetting the game
- The global keyword is used to indicate that the function will modify the global variables bird_y, bird_velocity, and pipes
- When called, reset_game resets the bird's vertical position (bird_y) to the middle of the screen, sets its velocity (bird_velocity) to 0, and clears the list of pipes (pipes)
- This section defines the main game loop, which is a continuous loop that runs the game logic and handles all the events
- The loop repeats over the events that have occurred since the last time using pygame.event.get ( ).
- If the event type is pygame.QUIT, the game quits, and exits the program
- If the event type is pygame.KEYDOWN and the pressed key is the spacebar (pygame.K_SPACE), the bird's velocity is increased by BIRD_JUMP. That simulates the bird jumping when the spacebar is pressed!
- These lines update the bird’s position based on gravity and handle game logic
- bird_velocity is increased by GRAVITY, simulating a downward acceleration due to gravity
Pipes are randomly generated with a 10% probability on each repetition of the loop, and their positions are tracked in the pipes list.
Existing pipes are moved to the left (pipe.x += PIPE_VELOCITY).
Pipes that go off-screen (x-coordinate less than or equal to 0) are removed from the pipes list.
Collisions are checked using bird_rect.colliderect(pipes) (collision with pipes) or if the bird goes out of the screen. If a collision is detected, the game is reset using the reset_game function.
- The screen is filled with a white color (screen.fill(WHITE)) to clear the previous frame
- The bird and pipes are drawn on the screen using the blit method
- The bird is created at a fixed horizontal position ((100, bird_y))
Each pipe is made at its current position.
clock.tick(FPS) is used to control the frame rate of the game loop. It makes it so that the game runs at a maximum of FPS frames per second.
dispatcher and osc_server are from the pythonosc library, used for handling OSC (Open Sound Control) messages.
This code imports the libraries I need.
K_SPACE and KEYDOWN are constants from the pygame.locals module, which I used to identify the spacebar key and keydown events.
This line initializes the Pygame library (again)! 😅
This part defines the IP address and OSC port of the Muse headband.
You’d need to replace ‘127.0.0.1’ with the actual IP address of your Muse headband, and 5000 with the OSC port used by your Muse headband.
- A Dispatcher object is created from the pythonosc library. I used it to map OSC paths to specific functions.
- dispatcher .map(“/muse/elements/blink”, print) maps the OSC path /muse/elements/blink to the built-in print function. Adjust this line to map the OSC path to a function that handles Muse data in your way.
- An OSC server is created using the ThreadingOSCUDPServer class from the pythonosc library
- It’s configured to listen for incoming Muse data on the specified IP address and port
- A message is printed so it indicates that the server is listening
Pygame is initialized again at the start of the game.
- This function (jump_bird ( )) makes the Flappy Bird jump
- It sets the global variable bird_velocity to BIRD_JUMP, which is a constant determining the upward velocity when the bird jumps
A Pygame window is created with dimensions (1, 1) This window is pretty small and is primarily used to handle events related to the spacebar key!
- This is the main event-handling loop
- It repeats events in the Pygame event queue using pygame.event.get( )
- If a KEYDOWN event is detected and the pressed key is the spacebar (K_SPACE), the jump_bird( ) function is called
I added the code server.handle_request( ) to handle incoming OSC messages from the Muse headband.
To also physically see how your brainwaves change with different actions and stream the brain activity, you can download the Mind Monitor App in the App Store for around 19$.
The first step is to connect your muse headband to the app by using Bluetooth:
Then you need to go to the settings, change the OSC Stream Target IP to the one on your computer and the OSC Stream Port to any number, like e.g. 8000.
Now you have to click on the little button that looks like the hotspot sign on the bottom right, so that your data will be streamed.
The last step is to observe your brainwaves and understand how they work!
You can see, that there are 4 different channels on the right, TP9, AF7, AF8, and TP10, which I also mentioned before, that showcase your electrical activity in the brain.
Playing Flappy Bird using my thoughts was a difficult challenge, since I’ve never coded before and learned Python in a span of 2 months. But, it all paid off in the end.
Thank you so much for your attention and I hope you liked my project! See you next time! 🧠
