Deploying a Testnet with OnFinality

A crucial step for all Polkadot Parachain hopefuls is to deploy a testnet. Testnets allow teams to test new code and ideas before launching them, showing proof of concept in a live environment without the risks of tying financial incentives to the chain.

For our testnet (Neumann), OAK utilized our infrastructure partner OnFinality. As with all things new and exciting, the process involved quite a bit of learning for our team. As part of our mission to support the ecosystem and deliver value at all stages of development, we have created this walkthrough guide to setting up your own testnet, including our step by step process as well as lessons we’ve learned along the way.

NOTE as of May 31, 2022: We have since deprecated Neumann testnet in lieu of Turing staging.

Please find more information on our docs here: https://avaprotocol.org/docs/networks/

The instructions below are still useful if you’d like to deploy a testnet.

Step 1: Spinning up a Relay Chain

All parachains using the Polkadot architecture require a relay chain to interact with (read more about chain types and their relationship here). OnFinality allows for this using their onf-cli command line tool or UI. The code blocks below details how we spun up our testnet relay chain, along with some guidance and links to OnFinality’s Github documentation:

# Install the onf-cli
sudo curl -s 
https://raw.githubusercontent.com/OnFinality-io/onf-cli/master/scripts/install/install.sh | sudo bash
# Setup with API keys (under Account tab)
# Note: If you’re switching workspaces, you’ll have to do the setup (you can use the same api keys) and the last prompt asks which workspace
onf setup
# Spin up the relay chain
# We are using OnFinality’s out of the box rococo relay chain configuration(which we keep in a folder called resources) for the relay testnet
cd resources
onf network bootstrap -f rococo-testnet-relaychain.yaml
# You will want to grab the bootNode p2p address from the output. This will be one of the last things printed.
# Example:Updated network spec with new bootnode list:
/dns4/node-6877053450743120–0.p2p.onfinality.io/tcp/28848/ws/p2p/12D34232pb3HEreJ1d423Tc15QW1gxYEse6564yAJ

Most of the magic here is in the onf-cli chain configuration file. Make sure to note in the example below which sections you will want to change for your own testnet.

yml
networkSpec:
 # Change the network names here to suit your project.
 
displayName: rococo-testnet-relaychain
 name: rococo-testnet
 protocol: substrate
 imageRepository: parity/polkadot
# You will want to update this to whatever version of polkadot you are using. I.E. what branch are all of your crates based on.  
imageVersion: v0.9.13
  config:
    nodeTypes:
      full:
        args:
          - key: “-force-authoring”
          - key: “-chain”
            file: rococo-testnet.json
      validator:
      args:
      - key: “-force-authoring”
      - key: “-chain”
        file: rococo-testnet.json
validator:
# After some discussion with the team at Polkadot, we found running four plus one validators per parachain was successful for our use case. Make sure to speak with other experienced projects to get a good idea for your ideal number.
count: 6
  sudoArgs:
#sudoArgs is used to reduce the setting of sudo users for each node. For this example, there will be 6 nodes running. The first node will be assigned alice, the second to bob,then repeating for all nodes. 
   - --alice
   - --bob
  node:
# Here you can update the node name to something relevant to your own project and the cli tool will automatically append the node number.
    nodeName: rococo-testnet-validator
    nodeType: validator
    clusterKey: jm
# Per OnFinality, make sure that you’re running at least 4 or more units of CPU here otherwise you can run into issues
    nodeSpec:
      key: unit
      multiplier: 4
    initFromBackup: false
    publicPort: true
    useApiKey: true
    storage: 30Gi
  sessionsKey:
# These are really full node: https://wiki.polkadot.network/docs/maintain-sync.
bootNode:
  count: 1
  node:
    nodeName: rococo-testnet-bootnode
    nodeType: full
    clusterKey: jm
    nodeSpec:
      key: unit
      multiplier: 4
    initFromBackup: false
    publicPort: true
    useApiKey: true
    storage: 30Gi

Step 2: Spinning up your Parachain

Now it’s time to get your code ready and spin up your parachain.

Creating the Chainspec

Before you can launch your parachain you will need to create your chainspec from node/src/chain_spec.rs. This is where you will change the genesis state for your parachain. Up until now you have probably used the default accounts for your chain (Alice and friends). For your testnet you will want to use real accounts. To do this you will need to update the following in your chain_spec.rs file.

1. Update Sudo to your account
2. Update the endowed account to you and your team.
3. Create your own accounts for the invulnerables. Each collator will use one of these accounts for aura. That means you need as many of these accounts as the number of initial collators you want to run -> use subkey to generate the accounts. ** Make sure to save all of their information. You will need it later.

Once you are done editing the chain_spec.rs run the following commands:

cargo build --release
./target/release/neumann-collator build-spec -disable-default-bootnode -chain=<your chain’s name> -raw > resources/<your testnet’s name>.json

This will create a new chainspec in json format for your chain. It is worth noting that generating the chain spec is non-deterministic. That means all of your nodes will need to use the file generated at this step, and not another version of it generated from the same executable.

** Make sure to use the executable you build here for all of the following steps. If you need to rebuild the executable then you will need to redo all the steps from here down.

The generation of this file is non-deterministic so make sure to save

** Additionally, check to ensure your chainspec defines its paraId in code and not as a runtime parameter.

Generate the Genesis State and Wasm

Now that you are done changing the source code you can run the following commands to generate the genesis state and wasm. You will need to upload these to your relay chain in a little bit. It is worth noting we have to pass in your chain to the export-genesis-state command, just like the build-spec command, as the chainspec will change the genesis state.

# Export genesis state
./target/release/neumann-collator export-genesis-state -chain=<your chain’s name> > genesis-state
# Export genesis wasm
./target/release/neumann-collator export-genesis-wasm > genesis-wasm

Create Your Image

Your last step of prep work is to create an image containing your latest substrate executable. When you deploy to ONF you will need to give it this image to deploy.

** Remember, you need to use the executable here that you built when creating the chainspec.

Create the Network

Now it’s time to create your network. Once the network is created you can spin up the nodes for your parachain on it. We used the ONF UI and not the onf-cli to do this because we wanted the flexibility of the UI as we created the parachain full nodes and collators.

1. Navigate to the networks tab
2. Click on “Create Network Spec”
3. For the first screen
   - Input your display name
   - Protocol = Polkadot Parachain
   - Input the docker repository where you saved your image, and pick the image you want to use (as a warning it can take a while for a newly pushed image to show up here). Make sure you select the right repository, as it cannot be changed.

For the “Full Node” (boot node) add the following:

• Parachain Arguments
  --force-authoring
--chain -> upload your chainspec file
• Relaychain Arguments (which is everything below the -- )
  --execution=wasm
—-chain -> upload the relay chain spec
  —-bootnodes -> you saved this earlier from the relay chain launch

For the “Collator” node add the following:

• Parachain Arguments
  --force-authoring
-chain -> upload your chainspec file
  --collator
--bootnodes-> You will get this value from your “Full Node” once it’s spun up.
  remove the --validator params
• Relaychain Arguments
  --execution=wasm
--chain -> upload the relay chain spec
  --bootnodes -> you saved this earlier from the relay chain launch

Launch the Full Node (Bootnode)

Now go ahead and launch a dedicated node (“Full Node”) on the network. It should be a “Full Node”. We usually use the same cloud provider and region as the relay chain, then once the node is up, get the “P2P” address from it. Make sure you update the --bootnodes Parachain argument for the collator in the Parachain network configuration.

Launch the Collators

You can now launch as many collators as makes sense for your testnet. Remember, you must have one collator per “invulnerable” account that you created and added to the chainspec.

Setup Keys in Your Parachain Nodes

Earlier we created some new accounts for the invulnerables in the chainspec. These accounts are used by aura in order to achieve consensus. In order for these accounts to actually be used we need to insert the keys for those accounts into the chain. To do this you need to make an RPC call to each collator (NOT the bootnode) in your parachain. You can get the RPC UI url in the details section for a node. See below for more information on the RPC call.

• Endpoint: author -> insertKey
  - keyType: aura
  - suri: <the secret phrase for the account>
  - publicKey: <the public key (SS58)>

Once you have activated an account for each node you are good to go!

** You can verify this worked by calling Endpoint: author -> hasKey

Register your Parachain

The first thing you need to do is create a parachain slot for 2000. This just means going to the Network -> Parachains -> Parathreads and clicking on the + ParaId button. Once you reserve the 2000 paraId you can register your parachain.

** We are assuming you set your paraId to 2000 in your chainspec. If you used a different number then you will need to create a slot for your paraId.

There are two ways to register your parachain.

Short Term Registration

If you are just going to use this testnet for a day you can use Developer -> Sudo -> paraSudoWrapper.sudoScheduleParaInitialize. This will register your parachain with the relay chain for a day then it will revert to a parathread.

Long Term Registration

If you plan on using this testnet for any real amount of time you will want to create a parathread then force a lease. In order to do this first create a new parathread for your blockchain. Once this is done we will use Developer -> Sudo -> slots.forceLease to register your parachain for however long you wish.

Step 3: Play with Your Parachain!

You will need to wait a few minutes for your Parachain to register, but once it starts producing blocks you are all set to start using your testnet. Have fun!