8 Commits

Author SHA1 Message Date
Twirre Meulenbelt
c634e4b516 chore: replace print with logging and make robot conditional
All print statements in the main program, and components used by the main program, have been replaced with appropriate logging statements. The connection to the robot now only gets made when it's possible, otherwise only the microphone will be run.

ref: N25B-119
2025-10-02 16:13:39 +02:00
Twirre Meulenbelt
2132a74321 fix: allow access to state's exit_event while exiting
When exiting, the state's `is_initialized` flag is unset. Noticeable on Windows, when a thread tried to access the state's `exit_event` property to check whether it had been set, it would complain that the state was no longer initialized. Now, even when no longer initialized, if the `exit_event` is set, it will not raise an error when accessing this attribute.

ref: N25B-119
2025-10-01 17:34:51 +02:00
Twirre Meulenbelt
d21c7fa423 fix: always use 1 audio channel
Before, I chose the number of audio channels that the microphone supports. Should be 1.

ref: N25B-119
2025-10-01 13:41:53 +02:00
Twirre Meulenbelt
afae6fc331 feat: stream audio to CB
Uses PyAudio and ZeroMQ to publish audio chunks.

ref: N25B-119
2025-10-01 10:50:53 +02:00
da99b5cd62 chore: update README 2025-09-30 13:26:42 +02:00
d48ea930a1 chore: complete installation instructions
ref: N25B-115
2025-09-30 13:16:33 +02:00
9e001da685 chore: update README and gitignore
Add installation instructions for the development environment.

ref: N25B-115
2025-09-30 13:14:52 +02:00
a41552f7c6 feat: basic implementation of CB2RI
Nothing fancy yet. When we receive a message through ZeroMQ's PUB/SUB
architecture, we tell the robot to say it out loud.
2025-09-27 20:41:03 +02:00
6 changed files with 273 additions and 65 deletions

121
README.md
View File

@@ -1,93 +1,84 @@
# PepperPlus-RI
## Development environment
### Linux (or WSL)
Start off by installing [Pyenv](https://github.com/pyenv/pyenv?tab=readme-ov-file#installation) and walk through the steps outlined there (be sure to also add it to PATH). Also install the [Python build requirements](https://github.com/pyenv/pyenv/wiki#suggested-build-environment). Afterwards, install Python 2.7 and activate it for your current shell:
## Getting started
To make it easy for you to get started with GitLab, here's a list of recommended next steps.
Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
## Add your files
- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
- [ ] [Add files using the command line](https://docs.gitlab.com/topics/git/add_files/#add-files-to-a-git-repository) or push an existing Git repository with the following command:
```
cd existing_repo
git remote add origin https://git.science.uu.nl/ics/sp/2025/n25b/pepperplus-ri.git
git branch -M main
git push -uf origin main
```bash
pyenv install 2.7
pyenv shell 2.7
```
## Integrate with your tools
You can check that this worked by typing
- [ ] [Set up project integrations](https://git.science.uu.nl/ics/sp/2025/n25b/pepperplus-ri/-/settings/integrations)
```bash
python -V
```
## Collaborate with your team
Which should return `Python 2.7.18`.
- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
- [ ] [Set auto-merge](https://docs.gitlab.com/user/project/merge_requests/auto_merge/)
Next, `cd` into this repository and create (and activate) a virtual environment:
## Test and Deploy
```bash
cd <path to project>/
python -m pip install virtualenv
python -m virtualenv .venv
source .venv/bin/activate
```
Use the built-in continuous integration in GitLab.
To be able to install the PyAudio Python package, you'll need to have the `portaudio` system package installed. On Debian or Ubuntu:
- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/)
- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing (SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
```shell
sudo apt install portaudio19-dev
```
***
Then you can install the required packages with
# Editing this README
```bash
pip install -r requirements.txt
```
When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thanks to [makeareadme.com](https://www.makeareadme.com/) for this template.
Now we need to install the NaoQi SDK into our virtual environment, which we need to do manually. Begin by downloading the SDK:
## Suggestions for a good README
```bash
wget https://community-static.aldebaran.com/resources/2.5.10/Python%20SDK/pynaoqi-python2.7-2.5.7.1-linux64.tar.gz
```
Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
Next, move into the `site-packages` directory and extract the file you just downloaded:
## Name
Choose a self-explaining name for your project.
```bash
cd .venv/lib/python2.7/site-packages/
tar xvfz <path to SDK>/pynaoqi-python2.7-2.5.7.1-linux64.tar.gz
rm <path to SDK>/pynaoqi-python2.7-2.5.7.1-linux64.tar.gz
```
## Description
Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
Lastly, we need to inform our virtual environment where to find our newly installed package:
## Badges
On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
```bash
echo <path to project>/.venv/lib/python2.7/site-packages/pynaoqi-python2.7-2.5.7.1-linux64/lib/python2.7/site-packages/ > pynaoqi-python2.7.pth
```
## Visuals
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
That's it! Verify that it works with
## Installation
Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
```bash
python -c "import qi; print(qi)"
```
## Usage
Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
You should now be able to run this project.
## Support
Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
### MacOS
## Roadmap
If you have ideas for releases in the future, it is a good idea to list them in the README.
On ARM CPU's, pyenv doesn't want to install Python 2. You can download and install it from [the Python website](https://www.python.org/downloads/release/python-2718/).
## Contributing
State if you are open to contributions and what your requirements are for accepting them.
Create a virtual environment as described in the Linux section.
For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
Then build `portaudio` for x86_64 CPU's.
You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
Then follow the remaining installation instructions in the Linux section.
## Authors and acknowledgment
Show your appreciation to those who have contributed to the project.
## Running
Assuming you have the virtual environment activated (`source .venv/bin/activate` on Linux) and that you have a virtual robot running on localhost you should be able to run this project by typing
## License
For open source projects, say how it is licensed.
```bash
python main.py --qi-url tcp://localhost:<port>
```
## Project status
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
where `<port>` is the port on which your robot is running.

69
main.py Normal file
View File

@@ -0,0 +1,69 @@
import sys
import logging
logging.
import zmq
from src.audio_streaming import AudioStreaming
from state import state
def say(session, message):
tts = session.service("ALTextToSpeech")
tts.say(message)
def listen_for_messages(session):
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://localhost:5556")
socket.setsockopt_string(zmq.SUBSCRIBE, u"") # u because Python 2 shenanigans
poller = zmq.Poller()
poller.register(socket, zmq.POLLIN)
logging.info("Listening for messages")
while not state.exit_event.is_set():
if not poller.poll(200): continue # At most 200 ms delay after CTRL+C
# We now know there's a message waiting for us
message = socket.recv_string()
logging.debug("Received message: {}".format(message))
if session: say(session, message)
def get_session():
if "--qi-url" not in sys.argv:
logging.info("No Qi URL argument given. Running in stand-alone mode.")
return None
try:
import qi
except ImportError:
logging.info("Unable to import qi. Running in stand-alone mode.")
return None
try:
app = qi.Application()
app.start()
return app.session
except RuntimeError:
logging.info("Unable to connect to the robot. Running in stand-alone mode.")
return None
def main():
session = get_session()
audio_streamer = AudioStreaming()
audio_streamer.run()
listen_for_messages(session) # Runs indefinitely, until CTRL+C
if __name__ == "__main__":
try:
state.initialize()
main()
finally:
state.deinitialize()

2
requirements.txt Normal file
View File

@@ -0,0 +1,2 @@
pyzmq<16
pyaudio<=0.2.11

0
src/__init__.py Normal file
View File

93
src/audio_streaming.py Normal file
View File

@@ -0,0 +1,93 @@
import threading
import pyaudio
import zmq
from state import state
def choose_mic_interactive(audio):
"""Choose a microphone to use. The `audio` parameter is an instance of PyAudio. Returns a dict."""
device_count = audio.get_device_count()
print("Found {} audio devices:".format(device_count))
for i in range(device_count):
print("- {}: {}".format(i, audio.get_device_info_by_index(i)["name"]))
microphone_index = None
while microphone_index is None:
chosen = input("Which device would you like to use?\n> ")
try:
chosen = int(chosen)
if chosen < 0 or chosen > device_count: raise ValueError()
microphone_index = chosen
except ValueError:
print("Please enter a number between 0 and {}".format(device_count))
chosen_microphone = audio.get_device_info_by_index(microphone_index)
print("Chose microphone \"{}\"".format(chosen_microphone["name"]))
return chosen_microphone
def choose_mic_default(audio):
"""Choose a microphone to use based on defaults. The `audio` parameter is a PyAudio. Returns a dict."""
default_device = audio.get_default_input_device_info()
return default_device
class AudioStreaming:
def __init__(self, port=5557):
self.port = port
self.audio = pyaudio.PyAudio()
self.microphone = choose_mic_default(self.audio)
self.thread = None
def run(self):
self.thread = threading.Thread(target=self._stream)
self.thread.start()
def wait_until_done(self):
if not self.thread: return
self.thread.join()
def _stream(self):
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:{}".format(self.port))
chunk = 512 # 320 at 16000 Hz is 20ms, 512 is required for Silero-VAD
stream = self.audio.open(
format=pyaudio.paFloat32,
channels=1,
rate=16000,
input=True,
input_device_index=self.microphone["index"],
frames_per_buffer=chunk,
)
try:
while not state.exit_event.is_set():
data = stream.read(chunk)
socket.send(data)
finally:
stream.stop_stream()
stream.close()
if __name__ == "__main__":
state.initialize()
try:
audio = AudioStreaming()
print("Starting audio streaming...")
audio.run()
import time
end = time.time() + 10
while not state.exit_event.is_set() and time.time() < end:
print "\rExiting in {:.2f} seconds".format(end - time.time()),
time.sleep(0.05)
state.exit_event.set()
audio.wait_until_done()
finally:
state.deinitialize()

53
state.py Normal file
View File

@@ -0,0 +1,53 @@
import logging
import signal
import threading
class State(object):
"""
Do not create an instance of this class directly: use the instance `state` below. This state must be initiated once,
probably when your program starts.
This class is used to share state between threads. For example, when the program is quit, that all threads can
detect this via the `exit_event` property being set.
"""
def __init__(self):
self.is_initialized = False
self.exit_event = None
def initialize(self):
if self.is_initialized:
logging.warn("Already initialized")
return
self.exit_event = threading.Event()
def handle_exit(_, __):
logging.info("Exiting.")
self.exit_event.set()
signal.signal(signal.SIGINT, handle_exit)
signal.signal(signal.SIGTERM, handle_exit)
self.is_initialized = True
def deinitialize(self):
if not self.is_initialized: return
self.is_initialized = False
def __getattribute__(self, name):
# Enforce that the state is initialized before accessing any property (aside from the basic ones)
if name in ("initialize", "deinitialize", "is_initialized", "__dict__", "__class__"):
return object.__getattribute__(self, name)
if not object.__getattribute__(self, "is_initialized"):
# Special case for the exit_event: if the event is set, return it without an error
if name == "exit_event":
exit_event = object.__getattribute__(self, "exit_event")
if exit_event and exit_event.is_set(): return exit_event
raise RuntimeError("State must be initialized before accessing '%s'" % name)
return object.__getattribute__(self, name)
# Must call `.initialize` before use
state = State()