mtsatellite/SETUP.md

SETUP MTSatellite

You will need a Minetest server with Redis support compiled in. Consult the Minetest documentation to figure out how to get such build. Furthermore you need the binaries mtdbconverter, mtseeder, mtredisalize and mtwebmapper in your PATH. Consult COMPILE how to build these.

Setting up MTSatellite takes six steps:

1. Backup your world
2. Convert world database into interleaved format
3. Start mtredisalize
4. Pre-compute the map tiles with mtseeder
5. Start the web server mtwebmapper
6. Configure and restart the Minetest server

Backup your world

MTSatellite is still young. So stop your running Minetest server and make a backup of your world before you will start crying.

Convert world database into interleaved format

MTSatellite operates best if the block data of the world is stored in a LevelDB database with a key scheme called interleaved. With this key scheme you can pick up sets of neighbored blocks a lot quicker than with a plain database. See Z-order curve at Wikipedia to grasp the core ideas. MTSatellite can run on plain LevelDB or SQLite3 world databases but this reduces the performance significantly. This is also not tested very well and will likely break your database. So do not use it! Stay with the interleaved format!

To convert your original plain SQLite3 or LevelDB database (Redis is not supported atm) to the interleaved LevelDB format you have to use mtdbconverter:

mtdbconverter -source-backend=sqlite /path/to/your/world/map.sqlite /path/to/your/world/map.db

Depending on the size of your world and the speed of your computer system this conversion will take some time. Change -source-backend=sqlite to -source-backend=leveldb if your world is stored as a LevelDB. mtdbconverter can also be used to convert your world back to the plain key scheme. Use mtdbconverter --help to see all options.

Start mtredisalize

mtredisalize is the component which serves the block data to Minetest and mtwebmapper as a Redis look-alike server. Start it with:

mtredisalize                               \
  -host=localhost                          \
  -interleaved=true                        \
  -change-url=http://localhost:8808/update \
  -change-duration=10s                     \
  /path/to/your/world/map.db

This binds the server to localhost port 6379 the default Redis port. You can shange it with the -port= option. The -interleaved=true option is mandatory if you use the interleaved format of the database. Forgetting it will end up in the crying mentioned above.
The -change-url= option is a forward reference to the mtwebmapper server which will be notified if the world has changed. If it is not configured the tile re-generation is not triggered. As long as the Minetest server is down there will be no changes and therefore it is safe to configure it even if the mtwebmapper service is not running.
The -change-duration= option specifies the amount of time how long the mtredisalize server should aggregate changes made to the world before reporting them to mtwebmapper. It defaults to 30 seconds but the value can be increased or decreased depending how often you want to update the map. Decreasing it will increase the computing pressure on your system so configure it wisely.

Pre-compute the map tiles with mtseeder

Even in a dynamical Mintest world played with many players most of the data is static over time. To generate a basic map to apply only changes to use mtseeder:

GOMAXPROCS=6 mtseeder              \
  -colors=/path/to/your/colors.txt \
  -output-dir=/path/to/your/map    \
  -workers=3

This contacts the mtredisalize server running at localhost port 6379 to fetch the block data from. You will need a colors.txt to map the block nodes to pixel colors of your map. The repository contains one.
If you want to have certain nodes to be transparent you can add -transparent=true to the options. In this case if the colors from colors.txt do have a forth color component with a numerical value lower than 255 (e.g 128) the corresponding pixels on the resultung tiles would be transparent.
See mtseeder --help for all options.

The -workers= option and the GOMAXPROCS= environment variable are completely optional but very useful to exploit multiple processor cores on your machine. Set GOMAXPROCS= to the result of nproc and -workers= to a number a little lesser. You have to experiment with this to find a good setting.

Even with good CPU usage generating the map and overview image tiles take a while.

Tip: A lot of the Minetest map tiles are white/empty but are saved as dupes in the file system. To deduplicate them you can use e.g. hardlink. You can also run it as a nightly cron job to dedupe the map on a regular basis.

Start the web server mtwebmapper

This web server serves the Leaflet compatibles to the browser and is contacted by mtredisalize if something in the world has changed. In this case the corresponding map tiles are re-generated in the background. To start mtwebmapper use:

GOMAXPROCS=3 mtwebmapper           \
  -colors=/path/to/your/colors.txt \
  -web-host=""                     \
  -map=/path/to/your/map           \
  -web=/path/to/your/static/web    \
  -redis-host=localhost            \
  -workers=2

For the colors= options applys the same as said above. You can also add -transparent=true for transparency as mentioned above. The web-host= is the interface the server ist listening on. "" means all interfaces. The port defaults to 8808. For a productive setup you may consider running it behind a reverse proxy.
-map= has to be the same path as used by mtseeder.
-web= is the path to the static web data (Leaflet, HTML, CSS, etc.). You can take it from the repository

To fetch the block data from the mtredisalize you have to use the option redis-host=. If you omit this then there will be no background job to re-generate the map. This is useful if you want to serve a map that is only generated once whith mtseeder.

To see all the available options use mtwebmapper --help.

The GOMAXPROCS=/-workers= setting has to be adjusted to your system capacity. Do not give to much ressources to this if you planning to run the mapping webserver on the same machine as the Minetest server. On the other hand assigning more cores to it definitely helps to boost up the performance.

Configure and restart the Minetest server

Now everything is in place and the only thing left ist to re-configure the Minetest server itself. You have to open your /path/to/your/world.mt file in your text editor and replace the backend with a Redis configuration:

backend = redis
redis_hash = IGNORED
redis_address = localhost

You may have to set redis_port too if you run mtredisalize not on port 6379.

Now we are all done and you can fire your Minetest server up again. :-)