Running an Archive Server

​

Recommended footprint

• 2 Archive Servers running MongoDB + monad-indexer
• 2 Archive Writer nodes, aka full node + monad-archiver
• Many full nodes serving RPC requests, connected to both Archive Servers for historical transactional data

​

Recommended Archive Server host specs

• CPU: 16 cores
• RAM: minimum 64GB; prefer >512GB for best performance
• Storage: minimum recommended 16TB; preferred 32TB+
  • NVMe SSD
  • RAID 10
  • Minimum 10K IOPS sustained
• Network: >1GbE, scale higher as needed when serving more RPC servers

​

Fresh installation

​

On your Archive Server:

1. Add the following section to /home/monad/.env
   # Replace <db username> and <db pwd> with your actual values
   DB_USER="<your db username>"
   DB_PWD="<your db pwd>"
   DB_VOLUME="~/archive-db-data"
   
   # Note: source and sink should generally be the same local MongoDB.
   # Indexer reads from:
   BLOCK_DATA_SOURCE="mongodb mongodb://${DB_USER}:${DB_PWD}@0.0.0.0:27017 archive-db"
   # Indexer writes to:
   ARCHIVE_SINK="mongodb mongodb://${DB_USER}:${DB_PWD}@0.0.0.0:27017 archive-db"
   OTEL_ENDPOINT="http://0.0.0.0:4317"
   
2. Run mongodb. Instructions show how to run in Docker, but operators can run however they choose.
   archive-db:
     image: mongo:latest
     command: mongod --bind_ip 0.0.0.0
     networks:
       - host
     environment:
       MONGO_INITDB_ROOT_USERNAME: ${DB_USER}
       MONGO_INITDB_ROOT_PASSWORD: ${DB_PWD}
     ports:
       - "27017:27017"
     volumes:
       - ${DB_VOLUME}:/data/db
     logging:
       driver: journald
       options:
         tag: "mongo"
   
3. Start the db
   # create directory from .env db volume
   source /home/monad/.env
   sudo mkdir -p $DB_VOLUME
   sudo chown -R monad:monad $DB_VOLUME
   sudo chmod 700 $DB_VOLUME
   
   docker compose up archive-db -d
   

​

On your Archive Writer host:

4. Add the following section to /home/monad/.env
   ###############################################################
   ################### Vars that do not change ###################
   ###############################################################
   # Replace with your Archive Server credentials
   ARCHIVE_DB_USER="<your db username>"
   ARCHIVE_DB_PWD="<your db pwd>"
   ARCHIVE_DB_HOST="<your db hostname>"
   
   ## Where block data gets written (your Archive Server)
   ARCHIVE_SINK="mongodb mongodb://${ARCHIVE_DB_USER}:${ARCHIVE_DB_PWD}@${ARCHIVE_DB_HOST}:27017 archive-db"
   ## Change this if your ledger folder is in a different location
   BFT_BLOCK_PATH=~/monad-bft/ledger
   OTEL_ENDPOINT=http://0.0.0.0:4317
   ## This can be significantly higher during backfill
   MAX_CONCURRENT_BLOCKS=50
   
   
   ###############################################################
   ####### Backfill from Genesis (initial configuration) #########
   ###############################################################
   BLOCK_DATA_SOURCE="aws testnet-can-004-0-aavn9ll 50" # testnet
   BLOCK_DATA_SOURCE="aws mainnet-deu-010-0 50" # mainnet
   ## Alternatively you can use your other ArchiveDB if it has already been backfilled
   ## If using another Archive Server, define variables for it and use them:
   #OTHER_DB_USER="<other db username>"
   #OTHER_DB_PWD="<other db pwd>"
   #OTHER_DB_HOST="<other db hostname>"
   #BLOCK_DATA_SOURCE="mongodb mongodb://${OTHER_DB_USER}:${OTHER_DB_PWD}@${OTHER_DB_HOST}:27017 archive-db"
   FALLBACK_BLOCK_DATA_SOURCE="aws testnet-can-004-0-aavn9ll-0 50" # testnet
   FALLBACK_BLOCK_DATA_SOURCE="aws mainnet-deu-010-0 50" # mainnet
   
   ###############################################################
   ###################### Normal Operation #######################
   ###############################################################
   ## Archiver checks triedb first for block data
   BLOCK_DATA_SOURCE="triedb /dev/triedb 5000"
   
   ### FALLBACK_BLOCK_DATA_SOURCE ###
   ## This is used whenever data is missing from BLOCK_DATA_SOURCE
   ## Normally used after state-sync
   ## If you have another Archive Server for redundancy, configure it here:
   #OTHER_DB_USER="<other db username>"
   #OTHER_DB_PWD="<other db pwd>"
   #OTHER_DB_HOST="<other db hostname>"
   #FALLBACK_BLOCK_DATA_SOURCE="mongodb mongodb://${OTHER_DB_USER}:${OTHER_DB_PWD}@${OTHER_DB_HOST}:27017 archive-db"
   
   ## Alternatively you can use category-labs aws bucket
   ## Note: YOU pay for S3 egress costs if pulling from this bucket
   FALLBACK_BLOCK_DATA_SOURCE="aws testnet-can-004-0-aavn9ll 50" # testnet
   FALLBACK_BLOCK_DATA_SOURCE="aws mainnet-deu-010-0 50" # mainnet
   
5. Backfilling:
   As of v0.12.3, --start-block is no longer supported as a daemon argument. Instead, use the set-start-block subcommand to set the starting block marker imperatively before starting the daemon. This prevents accidental progress loss from systemd overrides.
   # [Optional] Set a specific start block if needed
   # This is typically only needed for initial setup or recovery scenarios
   # Example: start archiving from block 1000000
   monad-archiver set-start-block --block 1000000 --archive-sink "${ARCHIVE_SINK}"
   
   # For async backfill marker, add the --async-backfill flag:
   # monad-archiver set-start-block --block 1000000 --archive-sink "${ARCHIVE_SINK}" --async-backfill
   
   # Start the archiver daemon
   sudo systemctl start monad-archiver
   # Should show it running
   # Double check the arguments look correct based on the above ^
   systemctl status monad-archiver
   # Expect many:
   # > INFO: Successfully archived block block_num=X
   journalctl -u monad-archiver -o cat
   
6. Once monad-archiver has caught up to the chain tip you will see:

   INFO: Nothing to process

7. This means backfilling is done and you should
   1. Comment out BACKFILL section in your .env and uncomment Normal Operation
   2. Restart: systemctl restart monad-archiver
   3. Verify status and logs as above

​

On your ArchiveDB host

8. Start monad-indexer
   sudo systemctl start monad-indexer
   systemctl status monad-indexer
   # Expect many:
   # > INFO: Indexing block...
   # > INFO: Index spot-check successful
   journalctl -u monad-indexer -o cat
   

​

[Optional] Set up monad-archive-checker

1. Add the following systemd override to monad-archive-checker
   sudo systemctl edit monad-archive-checker
   
   [Service]
   ExecStart=
   ExecStart=/usr/local/bin/monad-archive-checker \
       # Example --bucket my-org-mainnet-checker
       # Note: storing checker state in local mongo or filesystem
       #       will be supported in a future release
       --bucket <S3 Bucket for storing checker state>
       checker
       # Example of comparing a local self-hosted mongo against a Category Labs aws bucket
       # --init-replicas "<aws mainnet-deu-009-0 50,mongodb mongodb://<username>:<pwd>@<db hostname>:27017 archive-db>,mongodb mongodb://<username>:<pwd>@<second db hostname>:27017 archive-db>"
       --init-replicas "<ArchiveDB 1>,<ArchiveDB 2>"
   
2. Reload and Restart
   sudo systemctl daemon-reload
   sudo systemctl restart monad-archive-checker
   systemctl status monad-archive-checker
   
   # Check for errors
   journalctl -u monad-archive-checker -o cat -f