To set up SmokeDetector (SD), please execute the following commands:
git clone https://github.com/Charcoal-SE/SmokeDetector.git cd SmokeDetector git checkout deploy sudo pip3 install -r requirements.txt --upgrade pip3 install --user -r user_requirements.txt --upgrade
config.sample to a new file called
config, and edit the values required. If you are going to be running this SmokeDetector instance as a normal SD instance with metasmoke, see the “config” file with MS below.
You should now be able to run SmokeDetector.
You will need a keybase account. You will also need to have the “smoke detector runner” role on metasmoke (MS). Contact an MS admin to request the role. When given the role, you should separately be added to the “charcoal” channel on keybase.
In the files section for the keybase “charcoal” channel, there is a “config” file. You should use that config file as a basis for the “config” file you use in your “SmokeDetector” directory.
locationto the name you choose for your SD instance. This should be your username, followed by a slash and an identifier that’s unique among all your Smokey instances. This is the name that your instance will display in messages in chat and what will be displayed on MS for status.
If a file “rooms_custom.yml” exists in the SD directory, then it will be used instead of the “rooms.yml” file. If you are not running a full SD instance, which is reporting into Charcoal HQ and all other rooms, you will need to create a “rooms_custom.yml” which describes the rooms in which you want SD to be reporting and accepting commands. For more information about the format used in “rooms.yml” and “rooms_custom.yml”, please see: Rooms configuration.
At this point, you should be able to run SmokeDetector using the instructions below.
Using a AWS EC2 instance can be a quick way to get an SD instance up and running. Based on reports from @iBug in chat, a T2.micro instance, what’s available from the AWS Free Tier for 12 months, doesn’t have enough CPU credits, at 6/hour, for the instance to be the active SD instance full-time. Brief testing with a T3.micro, 12 CPU credits/hour, indicated that it’s not sufficient to run SD full-time without additional CPU charges. It’s expected that a T3.small, 24 CPU credits/hour, should be sufficient for a full-time active SD instance. However, a T2.micro instance does appear to be enough for SD to run as a standby instance to fill in on an irregular basis without CPU charges, assuming it’s not running for more than a couple/few hours. An additional note is that SD on the AWS Linux AMI described below did report some out of memory errors when running on a T3.micro (1GB) without any swap configured. On the other hand, SD functions on a AWS Lightsail instance with 512MB RAM and 2GB swap configured, but that Lightsail instance doesn’t have enough CPU credits to operate SD for more than a few hours (and more cannot be purchased). On yet another hand, a t4g.nano, which has the same 512MB of RAM and 2GB of swap, but 2 vCPU compared to 1 vCPU for the low-end Lightsail, can be configured in unlimited modewith respect to CPU Credit usage, so can run SD indefinitely, just with potential substantial CPU credit charges if running full-time. The T4g.nano can also be significantly less expensive (~50%) than the lowest-end Lightsail, if run as a standby SD instance and the T4g.nano has been purchased as a reserved instance. [Note: the numbers for needed CPU credits mentioned above for T2 and T3 instances are old information. SD’s code changed in 2022-05 which should have notably lowered the overall CPU usage and the above comparisons for T2 and T3 have not been updated.]
To set-up a AWS EC2 instance for SD you can do the following:
yum. Other versions may use
pacmanetc. In addition, other versions of Linux may already have versions of some of the packages installed in the commands below, or may be missing yet other packages. The commands listed below were only tested in the AMI mentioned above.
Execute the following commands. You can copy-&-paste all of them at one time. The commands above the blank line get your EC2 instance to where SmokeDetector can be installed (only tested in the AMI mentioned above). The commands below the blank line clone SmokeDetector from GitHub and install its Python dependencies (the commands below the blank line are a duplicate of those at the top of this page, but are included below to make copy-&-pasting easier):
sudo yum -y update sudo fallocate -l 2G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile sudo sh -c 'echo "/swapfile swap swap defaults 0 0" >> /etc/fstab' sudo yum -y install gcc sudo yum -y install git sudo yum -y install python37 sudo yum -y install python3-devel sudo python3 -m pip install pip --upgrade git clone https://github.com/Charcoal-SE/SmokeDetector.git cd SmokeDetector git config user.email "[email protected]" git config user.name "SmokeDetector" git checkout deploy sudo /usr/local/bin/pip3 install -r requirements.txt --upgrade pip3 install --user -r user_requirements.txt --upgrade
config.sampleto a new file called
config, and edit the values required.
To run, use
python3 nocrash.py, or
python3 nocrash.py standby (preferably in a daemon-able mode, like a
You can also use
python3 ws.py, but then SmokeDetector will be shut down after 6 hours;
when running from
ws.py will automatically be restarted.
(This is to be sure that closed WebSockets, if any, are reopened.)
nocrash.py is the controlling Python code for SmokeDetector. It runs
ws.py, which is the SD instance, in a subprocess and restarts the SD instance when it stops.
ws.py may stop as the result of various chat
!!/ commands, or do to errors. The syntax for
python3 nocrash.py [standby] [--loglevel=(debug|info|warning|error)] [no_se_activity_scan] [no_deletion_watcher] [no_edit_watcher]
standby— Starts the SmokeDetector instance in standby mode.
no_se_activity_scan— Causes the SmokeDetector instance not to watch the SE WebSocket for new posts, even when the instance is active. This gives you an SD instance which will listen for commands in chat, but won’t be scanning for new posts, other than ones you
!!/report. If you are not testing something which is directly involved with scanning posts from the SE WebSocket (i.e. you don’t need to scan a large number of posts, or test/use the SE WebSocket and per-site post fetching queues), then using
no_se_activity_scanwill probably be quite beneficial to you, as it will allow you to focus on what you are testing without seeing large amounts of output from scanning posts. This can also be used at times when the SE WebSocket is having problems.
no_deletion_watcher— Disables SmokeDetector’s
DeletionWatcher. The deletion watcher uses the SE WebSocket to watch for the deletion of posts which have been reported by SD. Using this in combination with
no_edit_watcherwill result in SD not opening the SE WebSocket, which can be helpful if the SE WebSocket is having problems, testing, etc. Using all those options will result in an SD instance which is monitoring chat for commands, which will allow performing chat commands without SD scanning SE sites based on any WebSocket reports.
no_edit_watcher— Disables SmokeDetector’s
EditWatcher. The edit watcher uses the SE WebSocket to watch for edits on posts which have been scanned by SD. Using this in combination with
no_deletion_watcherwill result in SD not opening the SE WebSocket, which can be helpful if the SE WebSocket is having problems, testing, etc. Using all those options will result in an SD instance which is monitoring chat for commands, which will allow performing chat commands without SD scanning SE sites based on any WebSocket reports.
--loglevel=(debug|info|warning|error)— Set the level of log output to send to the console. By default, there’s quite a bit of information logged to the console while scanning.
Portions of this page were taken from README.md.