I made a micro server to host my bots

8 min readMay 24, 2019

I love creating bots and writing automation scripts for web, I know other developers love that too. I have been using Google’s compute engine to run and host many of my chatbot apps. Some of the apps have simple automation logic that require very low memory and CPU power and do not utilize all the resources of the instance assigned for the month. Cloud platform are expensive, and very much for hosting side projects which have low resource consumption. Hence I made a server to deploy some simple bot apps written in JavaScript/NodeJS.

In this article, I will explain how I made a Raspberry Pi 3 server to host my bots. This is not a web server project. This system will do a bare minimum task of deployment. I have not configured RPi to run fancy Devops tools like containers, dockers, kubernetes etc. but they can be installed on the top of this setup.

Use Cases

It can run scripts to scrape web, fetch data over API endpoints and update settings on a remote server/PC. It can run multiple programs at the same time just like any server on various platforms. It is not a web server.

Features

Easy to setup
Inexpensive
Require low power

Hardware

Raspberry Pi 3 Model B

It is a single-board computer with wireless LAN and Bluetooth connectivity. It has a 1.4GHz processor with armV7 architecture. You can read more about the specs here. In RPi family this model is the latest, most advanced and most expensive. In my country it cost around 40 USD.

Micro SD Card

8GB SD card with transfer speed of 80MB/s or more, however I have used 16GB as the extra storage can be used for other RPi projects.

Peripherals

Monitor with HDMI support and a generic USB keyboard.

Flash Drive(optional)

Some files were transferred from my PC to RPi via a flash drive, it is not significant to setup procedure however one might need it, so having any kind of flash drive is a good idea.

Power Adapter

RPi3 uses a general mobile charger/power adapter. I found a spare one easily in my drawers.

Operating System Setup

First job is to install and setup the operating system. I am creating a system to deploy apps so there is no requirement for desktop GUI, in fact installing a full OS with pre installed utilities will increase RAM consumption and slow down the performance. So I have used Minimal Raspian OS without desktop, which is only 356MB in size. There are server distros that are specifically made to run web servers, they have programs like Apache, nginx etc. I don’t need them for my bot projects.

First, the OS files needs to be flashed on to the memory card. Use a card reader to plug memory card into the PC, then use a flashing software to flash the downloaded .zip file. I have used Balena Etcher. Once flashed, put the SD card into the card slot of RPi.

Now, all the things needed to operate RPi are connected. Connect monitor and keyboard to RPi. Now plug in the power adapter and switch on. If the OS file was not corrupt and it was flashed properly, it will boot and show a login screen which you will be able to see on the monitor.

After booting, OS will show a black terminal, a bunch of text and a blinking cursor. It is now asking to log into the OS by providing the credentials as shown in the image above. Type pi for username and raspberry for password. The default user is pi. Users can be created with useradd command which is an interesting thing to learn in linux/unix systems. However for the simplicity of this article we are just going to use the default pi user.

Once logged in, the system now need software setup. I am using NodeJS so the tools I use are node specific and some are according to my likings. Almost all platforms like python, ruby, php, go etc. have tools which can be installed and configured in a similar way.

Connecting to the Internet

I do not have a broadband connection, I connect to internet with a hotspot device which is generally faster and cheaper than broadband in my city. If you have a connection to the broadband, you can plug in the ethernet cable to RPi from modem/router.

If you have connection to WiFi only, you have to configure the wireless LAN settings. A simple way to do that in Raspbian is to run raspi-config program. Run it with sudo raspi-config select networking option, then select WiFi, enter the WiFi network name in UUID field and then enter the password(leave if not required).

If you have installed other OS like fedora, arch, vanilla debian etc. the Wifi setup needs to be done manually from root settings because there may not exist a wizard like raspi-config. You can find the instructions for that here.

Installing Tools

Editors, compilers, project managers, package managers, version managers are among few tools that are required to configure and setup apps on any server/system before deployment. I have installed tools that are required by NodeJS projects/ecosystem, which are as follows.

Vim(optional)

Raspbian comes with a pre installed editor nano, its excellent. If you do not want to be bothered by flavor of editing and just want to keep editing tasks simple, I suggest you continue with nano instead of vim. Vim can be installed via debian package manager sudo apt install vim .

Tmux

So, in interactive terminal apps like XTerm, iTerm, Gnome term etc. It is easy to work with multiple processes by creating new tabs and windows. But on GUI-less OS like we have here, interactive terminal apps cannot be installed. Tmux solve this problem, we can work and manage multiple tabs and sessions right from the command line. I suggest you read the manpages of tmux for better understanding.

Node Version Manager — nvm

With nvm multiple versions of NodeJS can be installed. So if some projects use older versions of node, there is no need to install and configure node every time for a new project, rather, they can be set to use project specific local versions with nvm. Version managers are available for almost all platforms and languages like for python there is pyenv, ruby has rvm, go has g etc. To install nvm visit github page for the instructions.

Git

Git will help in cloning the projects from github and also pulling latest changes. Git is not installed by default but it can be with apt package manager sudo apt install git.

Project Manager for Node — pm2

If the project is going to serve interactively on the web like a chatbot, twitter-bot etc. projects can utilize load balancing. pm2 is a project manager that can deploy multiple instances of node apps and balance traffic. I use the cluster feature of pm2 that deploy instances in a cluster. You can read more about that here. pm2 can be installed with npm i -g pm2.

Deploy

The RPi server setup is now complete, lets try to setup and deploy a project on it. I have created a twitter bot that fetch links and titles from hacker news for certain topics and randomly tweet one every 45 mins. You can find it on my github. To connect it with twitter, you’ll need credentials issued by twitter API.

Start a tmux session by typing tmux in bash terminal. If you are already in a tmux session create a new session by pressing CTRL+B and typing :new .
Clone the project with git, git clone https://github.com/ap4gh/twitter-bot.git .
Once cloned, move into the directory and install the dependencies npm i.
Now rename api_key.example.json mv api_keys.example.json api_keys.json. Open this file with vim and fill in the twitter API credentials.
Finally, start and test the app with npm start.
If everything is working, stop the running app with CTRL+C and deploy it, pm2 start app.js -i max.

And that is it. The project is now deployed and running. You can now disconnect the monitor from RPi. Make sure RPi is connected to the power and there is uninterrupted internet connection.

Deploying with pm2

Apps need not to be deployed with project managers all the time. But, directly running the app sometimes crash due to poor network conditions and un-handled errors. pm2 can reload the app during such crashes. So I recommended deploying with pm2. With pm2 you can also see the deployment logs and telemetry.

Deploy more than one project at a time

To deploy more projects you can allocate new tmux sessions. To create a new session press CTRL+B while in a tmux session and type :new . A new bash terminal opens up in which you can clone, test and deploy more projects.

Copying files from PC

I had API keys and bot tokens which were 50 characters long, I could have typed the entire bot token in about 5 minutes but I thought of saving it in a file and copying that to the RPi. I haven’t researched enough on how to copy files from PC to RPi via LAN. But the simplest way I intuit was with a flash drive.

When flash drive is plugged into RPi, it has to be mounted. The contents of the flash drive is made available at /media/usb. The entire procedure to mount USB flash drive in GNU/Linux OS is given here.

What did I learned

IaaS servers like GCP Compute Engine or AWS Elastic Compute 2 have similar interface. A project like this gives a simulation of real cloud computing server. One big difference is that we have remote access to the cloud servers via SSH. It will be another great project to setup SSH access to this RPi setup.

Fun projects with this server

A server like this can deploy all kinds of bots. I have my development discord bot Mr. Scott deployed on it. Instagram bots that are created with python selenium can be deployed on it. These bots consume very less resources because they do not serve any traffic. This project can be transformed into a real web server. Also, with few more tweaks we can make it a SSH remote server, a server like that can store some information that can be accessed from anywhere in the world, we can call it our SSH drive much like Google Drive or Dropbox.

Thanks for reading.