PMFs: data, charts, and the coming galactic player wars...

Hi BGS'ers and ED CMDRs in general...

I am happy to announce the soft-launch of the AEDC Cartography Department Twitter bot. If you like the monthly "state of the human populated bubble" updates, you're probably going to like this. Through this bot, you can get most of these charts on-demand, whenever you want, up to date with the latest eddb/api data dump (within a few hours)


It is working, though I will be tinkering with it a bit more today (put the script properly in the background, add a few finishing touches, log properly etc.), so there may be a few late replies in the next couple of hours, but it is otherwise fully functional. The bot will reply and in general it's working.

Note: give it a little time to reply - the process is actually remarkably involved - if the bot is unable to produce the chart, it will let you know.

Here's how it works: tweet at the bot with a request. There is a format (see below), but it is basically based on a CMD SUBCMD structure. Doesn't matter where in the tweet, and you can add words to make it more personal ;) just make sure the two key words are space separated. So something like "hey @AedcCartography please map Alliance for me". Upper or lowercase also doesn't matter.

The bot replies with a basic usage example, but the following list will help get it right first time.

Supported commands/charts/maps:

bubble pmf

chart govtypes
chart govtypes-allegiance
chart govtypes-allegiance-systems
chart states
chart stats-totals
chart stats-normalized

histo pmf
histo pmfallegiance
histo all3plus

map bubble
map alliance
map empire
map federation
map superpowers (NEW!)
map alliance-assets
map empire-assets
map federation-assets
map independent-assets
map pmfhomes
map largefactions
map mediumfactions

The charts may not look quite as good as the monthly datasheets, where screencaptures are made directly from the SVG. This has some automated conversion behind it, and the results are not always quite the same - the histograms, for instance, have wonky text labels. They also don't contain any embedded images within the SVG as <image> tags error out. Otherwise, the charts and graphs should look quite familiar.

Enjoy!
 
Last edited:
A few technical details :) Not least because of D. Braben's connection to the Raspberry Pi Foundation.

This altogether uses 7 RasPis, mostly 3B+ and 2 RasPi 4.

Source: https://imgur.com/gallery/SkARdXq


The core work is done by this cluster: the one furthest left runs Postgresql with the database file running on an external SSD drive. This collects and stores data from eddb.io/api after downloading the daily dumps. The remaining 5 are a k3s cluster, a light-weight Kubernetes distribution, and serves the HTML, and gets the data from the database in order to produce the SVG charts and maps. One of the k3s worker nodes is a RasPi 4 after an original 3B+ somehow failed to boot, despite a new memory card. The cluster actually likes the extra power

A second RasPi 4 runs the twitter bot, which itself has to go through multiple stages:
  • interpret the command what chart/map to create
  • create a headless browser to request the map, and render the SVG
  • extract the SVG from the response and save to disk
  • convert to PNG
  • attach to tweet
Scripts are a combination of SQL, bash, Python, and the SVG maps with a lot of JavaScript

Could this have been done simpler, and not requiring quite as much infrastructure? Sure! :) But what would have been the fun in that ...
 
A few technical details :) Not least because of D. Braben's connection to the Raspberry Pi Foundation.

WOW! Mangal that is SO COOL!
Please for us oldies call it Orac?
Orac_episode.jpg
 
A few technical details :) Not least because of D. Braben's connection to the Raspberry Pi Foundation.

This altogether uses 7 RasPis, mostly 3B+ and 2 RasPi 4.

Source: https://imgur.com/gallery/SkARdXq


The core work is done by this cluster: the one furthest left runs Postgresql with the database file running on an external SSD drive. This collects and stores data from eddb.io/api after downloading the daily dumps. The remaining 5 are a k3s cluster, a light-weight Kubernetes distribution, and serves the HTML, and gets the data from the database in order to produce the SVG charts and maps. One of the k3s worker nodes is a RasPi 4 after an original 3B+ somehow failed to boot, despite a new memory card. The cluster actually likes the extra power

A second RasPi 4 runs the twitter bot, which itself has to go through multiple stages:
  • interpret the command what chart/map to create
  • create a headless browser to request the map, and render the SVG
  • extract the SVG from the response and save to disk
  • convert to PNG
  • attach to tweet
Scripts are a combination of SQL, bash, Python, and the SVG maps with a lot of JavaScript

Could this have been done simpler, and not requiring quite as much infrastructure? Sure! :) But what would have been the fun in that ...
Can't say much more than...
1606728631955.png
 
Top Bottom