mtsatellite/SETUP.md

# SETUP MTSatellite

You will need a Minetest server with Redis support compiled in. Consult the Minetest documentation to figure out how to get such build.
Furthermore you need the binaries `mtdbconverter`, `mtseeder`, `mtredisalize` and `mtwebmapper` in your **PATH**.
Consult [COMPILE](https://bitbucket.org/s_l_teichmann/mtsatellite/src/default/COMPILE.md) how to build these.

Setting up MTSatellite takes six steps:

1. [Backup your world](#markdown-header-backup-your-world)
2. [Convert world database into interleaved format](#markdown-header-convert-world-database-into-interleaved-format)
3. [Start `mtredisalize`](#markdown-header-start-mtredisalize)
4. [Pre-compute the map tiles with `mtseeder`](#markdown-header-pre-compute-the-map-tiles-with-mtseeder)
5. [Start the web server `mtwebmapper`](#markdown-header-start-the-web-server-mtwebmapper)
6. [Configure and restart the Minetest server](#markdown-header-configure-and-restart-the-minetest-server)

Experimental: Optionally you can [enable on map tracking of logged in players](#markdown-header-enable-on-map-tracking-of-logged-in-players).

## Backup your world

Stop your running Minetest server and make a backup of your world
before you will start crying.

## Convert world database into interleaved format

MTSatellite operates best if the block data of the world is stored in a LevelDB database with
a key scheme called interleaved. With this key scheme you can pick up sets of neighbored blocks a
lot quicker than with a plain database.
See [Z-order curve](http://en.wikipedia.org/wiki/Z-order_curve) at Wikipedia to grasp the core ideas.
MTSatellite can run on plain LevelDB or SQLite3 world databases but with slightly reduced performance.
This should work but to our knowledge it is not used in productive setups.
Stay with the interleaved format!

To convert your original plain SQLite3 or LevelDB database (Redis is not supported atm) to the interleaved
LevelDB format you have to use `mtdbconverter`:

    mtdbconverter -source-backend=sqlite /path/to/your/world/map.sqlite /path/to/your/world/map.db

Depending on the size of your world and the speed of your computer system this conversion will take some time.
Change `-source-backend=sqlite` to `-source-backend=leveldb` if your world is stored as a LevelDB.
`mtdbconverter` can also be used to convert your world back to the plain key scheme.
Use `mtdbconverter --help` to see all options.

You can skip the conversion if you want to use a plain database.

## Start mtredisalize

`mtredisalize` is the component which serves the block data to Minetest and `mtwebmapper` as a Redis
look-alike server. Start it with:

    mtredisalize                               \
      -host=localhost                          \
      -interleaved=true                        \
      -change-url=http://localhost:8808/update \
      -change-duration=10s                     \
      /path/to/your/world/map.db

This binds the server to localhost port 6379 the default Redis port. You can shange it with the `-port=` option.
The `-interleaved=true` option is **mandatory** if you use the interleaved format of the database. Forgetting it
will end up in the crying mentioned above. Set this flag to `false` if you are using a plain database.  
The `-change-url=` option is a forward reference to the `mtwebmapper` server which will be notified if the
world has changed. If it is not configured the tile re-generation is not triggered. As long as the Minetest server
is down there will be no changes and therefore it is safe to configure it even if the `mtwebmapper` service is not
running.  
The `-change-duration=` option specifies the amount of time how long the `mtredisalize` server should aggregate
changes made to the world before reporting them to `mtwebmapper`. It defaults to 30 seconds but the value can
be increased or decreased depending how often you want to update the map. Decreasing it will increase the
computing pressure on your system so configure it wisely.


## Pre-compute the map tiles with mtseeder

Even in a dynamical Mintest world played with many players most of the data is static over time. To generate
a basic map to apply only changes to use `mtseeder`:

    GOMAXPROCS=6 mtseeder              \
      -colors=/path/to/your/colors.txt \
      -output-dir=/path/to/your/map    \
      -workers=3

This contacts the `mtredisalize` server running at localhost port 6379 to fetch the block data from. You will
need a `colors.txt` to map the block nodes to pixel colors of your map. The repository contains a
[prefabricated](https://bitbucket.org/s_l_teichmann/mtsatellite/raw/default/colors.txt) or you can create
an adjusted one fitting your server with [mtautocolors](https://bitbucket.org/s_l_teichmann/mtautocolors).  
If you want to have certain nodes to be transparent you can add `-transparent=true` to the
options. In this case if a color from colors.txt does have a forth color component the numerical
value between 0 (fully transparent) and 255 (fully opaque) will be the base transparency of the
pixel. Every depth meter of the same material will reduce the transparency by 2%. This can be adjusted
with the `-transparent-dim=percent` flags.  
See `mtseeder --help` for all options. 

The `-workers=` option and the `GOMAXPROCS=` environment variable are completely optional but very useful
to exploit multiple processor cores on your machine. Set `GOMAXPROCS=` to the result of `nproc` and `-workers=`
to a number a little lesser. You have to experiment with this to find a good setting.

Even with good CPU usage generating the map and overview image tiles take a while.

Tip: A lot of the Minetest map tiles are white/empty but are saved as dupes in the file system. To
deduplicate them you can use e.g. [hardlink](https://bitbucket.org/s_l_teichmann/hardlink). You
can also run it as a nightly cron job to dedupe the map on a regular basis.


## Start the web server mtwebmapper

This web server serves the Leaflet compatibles to the browser and is contacted by `mtredisalize`
if something in the world has changed. In this case the corresponding map tiles are re-generated
in the background. To start `mtwebmapper` use:

    GOMAXPROCS=3 mtwebmapper           \
      -colors=/path/to/your/colors.txt \
      -web-host=""                     \
      -map=/path/to/your/map           \
      -web=/path/to/your/static/web    \
      -redis-host=localhost            \
      -workers=2                       \
      -websockets=false

For the `colors=` options applys the same as said above. You can also add
`-transparent=true` for transparency as mentioned above. The `web-host=` is the interface the
server ist listening on. `""` means all interfaces. The port defaults to 8808.
For a productive setup you may consider running it behind a reverse proxy.  
`-map=` has to be the same path as used by `mtseeder`.  
`-web=` is the path to the static web data (Leaflet, HTML, CSS, etc.). You can take it
from the [repository](https://bitbucket.org/s_l_teichmann/mtsatellite/src/default/cmd/mtwebmapper/web/)

To fetch the block data from the `mtredisalize` you have to use the option `redis-host=`. If
you omit this then there will be no background job to re-generate the map. This is useful
if you want to serve a map that is only generated once whith `mtseeder`.

To see all the available options use `mtwebmapper --help`.

The `GOMAXPROCS=`/`-workers=` setting has to be adjusted to your system capacity. Do not
give to much ressources to this if you planning to run the mapping webserver on the
same machine as the Minetest server. On the other hand assigning more cores to it definitely
helps to boost up the performance.

Setting the `-websockets=true` flag enables websocket support for the server. With this
feature turned on and changing the line (in `web/index.html`) from

    var useWebsocket = false; // Set to true if you want websocket support

to

    var useWebsocket = true; // Set to true if you want websocket support

the web client gets an extra 'auto update' button. When switched on the server
informs the client if something in the maps has changed. The displayed map will
then update automatically without the need of manual pressing the 'update view'
button. Of cause your browser needs Websocket support, too.

## Configure and restart the Minetest server

Now everything is in place and the only thing left ist to re-configure the Minetest server
itself. You have to open your `/path/to/your/world.mt` file in your text editor and replace the
backend with a Redis configuration:

    backend = redis
    redis_hash = IGNORED
    redis_address = localhost

You may have to set `redis_port` too if you run `mtredisalize` not on port 6379.

Now we are all done and you can fire your Minetest server up again. :-)

## Enable on map tracking of logged in players
MTSatellite can display logged in players on the map.  
This is an experimental feature and its only confirmed working on GNU/Linux systems.
OS X and \*BSD should work, too.

To use it install the [track_players](https://bitbucket.org/s_l_teichmann/mtsatellite/src/default/mods/track_players)
mod. Simple add a checkout to your mods folder and activate it in your world.mt file.

    ...
    load_mod_track_players = true
    ...

This minetest mod writes players position to a [named pipe aka FIFO](http://en.wikipedia.org/wiki/Named_pipe).
`mtwebmapper` is able to read from this file and serve these positions as GeoJSON to the browser.
The FIFO has to be created _before_ the start of the minetest server.

    $ mkfifo /tmp/mt_players_fifo

The path to the FIFO can be changed in track_players/init.lua

    ...
    local fifo_path = "/tmp/mt_players_fifo"
    ...

To use the feature in `mtwebmapper` add the argument `-players=/tmp/mt_players_fifo` to the list
of command line arguments.

*Caution*: Please start `mtwebmapper` before the minetest server! Caused by the nature of FIFOs and the
single threaded execution of minetest mods the minetest server will block if there is no consumer
reading the player positions.

The player tracking is well integrated with the websocket support. If you enable websockets you will
be able to see the players moving on the map.