Table of Contents

1. Natural deploy

NaturalDeploy.png

2. Motivation

Have you ever felt when using any declarative type of program that is used for deployment that it's really unnatural to use (i.e you are a YAML developer). This is in comparison to using any programming language (Which has advanced static type checkers, error diagnosis etc…).

The 2nd issue is that it's really hard to setup servers and it only works with nodes which have public IPs. Most orchestration tools are built on the basis to use static IPs and fixed ports.

The effect of this move is that you can easily get vendor locked and you are less likely to choose the economical choice of owning your own hardware. We have lesser gratefullness of how complex problems were abstracted in simple tools like FTP, GIT, OrgMode or the design principles of Unix.

The aim of "Natural deploy" is to be a library which builds on top of P2PRC to make it as easy as running a binary to deloy your custom tasks to your servers. This will gives you the flexibility to truly customise to build your own orchestration layer to run your programs across your cluster of servers which can be situated anywhere.

3. How does it work ?

overview.drawio.png

Figure 1: High overview of Natural deploy

The diagram above is a high level overview of the architecture in figure 1. What's important to note is that there are 3 nodes. The root node which is incharge of using the Proxy implementation from P2PRC to help nodes escape NAT.

You will notice there is 2 other nodes. Node 1 which is the process manager node for deploying tasks across to node 2 (We will call it a worker node). In reality you can have more than 1 worker node. The task is sent over via an SSH connection which uses P2PRC to map the port to escape NAT. The SSH public key can be manually added to the worker node "authorizedkeys" file. Or else you can play around with P2PRC "BareMetal" mode (There is no official documentation on this yet. Since it's still on the alpha stage). The idea of the baremetal mode is to automatically propogate to the task machines public keys to nodes in your P2PRC network which run as "BareMetal".

The task and worker nodes can be enabled by just running just a single binary. P2PRC is also designed as library based P2P network which makes it modular by design. This means if you want to expand your exsisting application to compile and self-replicate itself on another worker node. You can embed these rules within your application source code instead of having a seperate orchestrator program like Kubernetes and makes you sane by calling functions rather than becoming a "YAML" developer.

Another keypoint to emphasise is that programmers need to rely less on tools like Docker and so we make it simpler to run your program as a single binary since it's ideal to track the process. To kill your task it's as simple as pkill which runs on the kernel you installed on the machine you own.

4. How to install

The only requirement if you are compiling from source is to have the Go compiler. We will show this example by demonstrating a simple example which consists of 3 computers:

4.1. General instruction to run

It's important to use the P2PRC master branch for this library.

go get github.com/Akilan1999/p2p-rendering-computation@master

4.2. The first computer (Root node):

Which is has a static public address. This is required to traverse traffic for your computers behind NAT.

  • Create a file called root.go
  • Copy the following below into root.go
package main

import (
    "fmt"
    natural_deploy "github.com/Akilan1999/Natural-deploy"
    "github.com/Akilan1999/p2p-rendering-computation/client"
    "os"
)

func main() {

    // Checking if the lock file exists
    // if not configures the root node
    _, err := os.Stat("Root.lock")
    if err != nil {
        // Create the root node configuration. Function parameter (Node name
        // and port number)
        rootNode, err := natural_deploy.CreateRootNode("RootNode","3244")
        if err != nil {
            fmt.Println(err)
            os.Exit(1)
        }

        fmt.Println("----- Root Node information -------")
        client.PrettyPrint(rootNode)
        fmt.Println("-----------------------------------")
        fmt.Println("Daemon starting .....")

        os.Create("Root.lock")
    }

    err = natural_deploy.RunDaemon()
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
        return
    }
    fmt.Println("Daemon started .....")

    for {

    }
}
  • Compile root.go
go build root.go
# Run the binary as a background process
./root &
  • Store the output

    {
       "Name": "RootNode",
       "MachineUsername": "",
       "IPV4": "<public ipv4 address>", // The public address of your root server
       "IPV6": "",
       "Latency": 0,
       "Download": 0,
       "Upload": 0,
       "ServerPort": "3244", // The default port of the root node 
       "UDPServerPort": "",
       "BareMetalSSHPort": "22",
       "NAT": false,
       "EscapeImplementation": "",
       "ProxyServer": false,
       "UnSafeMode": false,
       "PublicKey": "",
       "CustomInformation": ""
    }
    

4.3. The 2nd computer (The server behind NAT)

This could for instance be an old laptop which you would want to bring back to life.

  • Create the file server.go
  • Add the following to the server.go file. We are going call this server "Test-Server".
  package main

import (
        "fmt"
        natural_deploy "github.com/Akilan1999/Natural-deploy"
        "os"
)

func main() {
        _, err := os.Stat("Regular.lock")
        if err != nil {
                // Create the root node to be added
                var rootnode natural_deploy.RootNode
                // make sure to fill in "<>" area with the appropriate information
                rootnode.IPAddress = "<public ipv4 address>"
                rootnode.Port = "<ServerPort>" // (i.e 8088 by default) 

                err := natural_deploy.CreateRegularNode("Test-Server", &rootnode)
                if err != nil {
                        fmt.Println(err)
                        os.Exit(1)
                        return
                }
                os.Create("Regular.lock")
        }

        // Start the daemon
        err = natural_deploy.RunDaemon()
        if err != nil {
                fmt.Println(err)
                os.Exit(1)
                return
        }

        for {

        }
}
  • Compile server.go

    go build server.go
    # Run the binary as a background process
    ./server &
    

4.4. The 3rd Computer (Task machine)

This can be your current laptop which can remotely deploy tasks to your self hosted server. We are going to run a simple python server on the remote machine called "Test-Server". We will start the task ,which will deploy the python server from our local machine to the "Test-Server" machine ,which will then start the server and will internally call P2PRC function calls to provide an static ip and port number from the Root node. Therefore escaping NAT which allows you to self host.

  • Create the deploy script which we will call "test.sh" (The python server runs locally in 8000).
nohup python3 -m http.server 8000 > server.log 2>&1 &
  • Create a script to kill the deployed python server. We will call this "kill.sh".
kill -9 $(lsof -t -i:8000)
  • Create the file task.go
  • Add the following to task.go
  package main

import (
    "fmt"
    natural_deploy "github.com/Akilan1999/Natural-deploy"
    "github.com/Akilan1999/p2p-rendering-computation/client"
    "github.com/Akilan1999/p2p-rendering-computation/p2p"
    "os"
    "time"
)

func main() {
    _, err := os.Stat("Task.lock")
    if err != nil {
        // If you want to add a custom root node
        var rootnode natural_deploy.RootNode
        // make sure to fill in "<>" area with the appropriate information
        rootnode.IPAddress = "<public ipv4 address>"
        rootnode.Port = "<ServerPort>"
        // Name of the task machine
        natural_deploy.CreateTaskMachine("Task", &rootnode) 
    }

    // Create a 5 second delay for mapping port
    time.Sleep(5 * time.Second)

    // ------------------- Create a task -----------------
    var task natural_deploy.Task
    task.Name = "Test"
    task.TaskFile = "test.sh"
    task.KillTaskFile = "kill.sh"

    // Get all node information
    table, err := p2p.ReadIpTable()
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
        return
    }

    fmt.Println("IP table read")

    // Searching for the information of a particular
    // node information (In this example Test-2 node)
    var machineInfo p2p.IpAddress
    for i, _ := range table.IpAddress {
        if table.IpAddress[i].Name == "Test-1" {
            machineInfo = table.IpAddress[i]
        }
    }

    fmt.Println(machineInfo)

    // Allocate ports
    var port client.ResponseMAPPort
    port.PortNo = "8000"
    task.ExposedPorts = append(task.ExposedPorts, &port)

    task.NodeInfo = &machineInfo
    // ---------------------------------------------------------

    err = task.CreateTask()
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
        return
    }

    // Create a 10 second delay from mapping port
    time.Sleep(10 * time.Second)

    // Print the task out
    task.PrintTask()

    // Below is to kill the task
    err = task.KillTask()
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
        return
    }

}

  • Compile server.go

    go build task.go
    # Run the binary as a background process
    ./task &
    

5. Creating a task through an example

FileStructure.drawio.png

Figure 2: Structure of a task

The diagram above in figure 2 shows the attributes required for a task and the it's structure when deployed to a worker node. This is created by an example which is deploying a simple static webpage and it's also how the current webpage is deployed.

Let's look at the first box on the top which consists of 3 files we are going to send over via a file transfer using a SSH connection. The inner rectangle consists of the "./deploy" binary which will send itself and 3 files to the worker machine. This means the machine running the "./deploy" binary is the task machine.

When a task is deployed at the first time, refer to the 2nd box which is node 2. A folder called "p2prc-task/" with a sub folder consisting of the task name is created in the home directory. This means apart from a SSH connection nothing else is needed for deploying a task.

The flags shown in figure 2 will build on the code snippets as shown below based on this file:

- Deploy.go (Source code to deploy)
- compile.sh (Cross compile to linux -> deploy.go)
- run.sh (Instruction of the remote machine to start the process)
- exit.sh (Instuction to kill on the remote machine the task)
- Index.html, style.css, <diagrams / pictures> (Static assets to transfer over)

The listing 1 shows the main files required to deploy the following documentation webpage.

-d string
        the directory of static file to host (default ".")
-daemon
        Starts the background process to learn about nodes in the network
-deploy
        Deploy the webpage to the designated machine
-kill
        Kills the deployed task
-ls
        List processes
-p string
        port to serve on (default "8034")
-update
        Updates the documentation content
func Daemon() error {
    _, err := os.Stat("Task.lock")
    if err != nil {
        err := natural_deploy.CreateTaskMachine("<Your worker machine name>", nil)
        if err != nil {
            return err
        }

        os.Create("Task.lock")
    }

    err = natural_deploy.RunDaemon()
    if err != nil {
        return err
    }

    return nil
}

The listing 3 shows the only setup needed in the worker node to run it as background process or daemon to broadcast itself in a zero configuration manner to the other task and worker nodes. There is lock file created to ensure once this configurations is set to start daemon directly.

// Name: Name of the task 
func Deploy(Name string) error {
    // Creating the task type
    var task natural_deploy.Task

    // Setting the name of the task
    task.Name = Name

    // Points to start the process on the worker node
    task.TaskFile = "run.sh"

    // Points to kill the process on the worker node
    task.KillTaskFile = "exit.sh"

    // To the worker machine you want to deploy the task in
    task.DeployMachine = "<Your worker machine name>"

    var err error

    // Allocate ports
    var port natural_deploy.Ports
    port.Port = "<TCP port in which your task will run>"
    port.DomainName = "<Domain name you want to exposed your task TCP port to>"
    task.ExposedPorts = append(task.ExposedPorts, &port)


    // ------------------------------------------------------------------------------
    // ------------------------------------------------------------------------------

    // In this example this binary will cross compile 
    // to the linux x86 architecture and deploy itself to worker node
    // as server running on the <TCP port in which your task will run>. 

    // Check if the linux binary to run is compiled
    _, err = os.Stat("Deploy-Linux")
    if err != nil {
        err = SelfCompileLinux()
        if err != nil {
            return err
        }
    }

    // ------------------------------------------------------------------------------
    // ------------------------------------------------------------------------------

    // Sends files for setup
    err = task.SendFile("<path of file you would want to send over>")
    if err != nil {
        return err
    }

    .... n number of files to send

    // Creates and runs the task on the worker node
    err = task.CreateTask()
    if err != nil {
        return err
    }

    // Print out the processes created
    ListProcess()

    return nil
}
// Name: <Task name>
func Update(Name string) error {
    task, avaliable := natural_deploy.ViewTasks(<Task name>)
    if !avaliable {
        return errors.New("task not found")
    }

    // Send the files you need to update or a new file
    // to send over.
    err := task.SendFile("<path of file you would want to send over>")
    if err != nil {
        return err
    }

    .... n number of files to send

    return nil
}
// Name: <Task name>
func KillProcess(Name string) error {
    task, avaliable := natural_deploy.ViewTasks(Name)
    if !avaliable {
        return errors.New("task not found")
    }

    err := task.KillTask()
    if err != nil {
        return err
    }

    return nil
}
func main() {
    // ------------------------- Defining the flags --------------------------------
    deploy := flag.Bool("deploy", false, "Deploy the webpage to the designated machine")
    kill := flag.Bool("kill", false, "Kills the deployed task")
    daemon := flag.Bool("daemon", false, "Starts the background process to learn about nodes in the network")
    update := flag.Bool("update", false, "Updates the documentation content")
    ls := flag.Bool("ls", false, "List processes")
    port := flag.String("p", "8034", "port to serve on")
    directory := flag.String("d", ".", "the directory of static file to host")
    flag.Parse()
    // ------------------------------------------------------------------------------

    // ------------------------ Handlers for the flags ------------------------------

    if *deploy {
        err := Deploy("<task name>")
        if err != nil {
            fmt.Println(err)
            return
        }
        return
    }

    if *update {
        err := Update("<task name>")
        if err != nil {
            fmt.Println(err)
            return
        }
        return
    }

    if *kill {
        err := KillProcess("<task name>")
        if err != nil {
            fmt.Println(err)
            return
        }
        return
    }

    if *daemon {
        err := Daemon()
        if err != nil {
            fmt.Println(err)
            return
        }
        for {

        }
    }

    if *ls {
        ListProcess()
        return
    }

    // ------------------------------------------------------------------------------

    // -------- part of the code executed on the task machine ------------

    http.Handle("/", http.FileServer(http.Dir(*directory)))

    log.Printf("Serving %s on HTTP port: %s\n", *directory, *port)
    log.Fatal(http.ListenAndServe(":"+*port, nil))

    // -------------------------------------------------------------------
}

Author: Akilan

Created: 2026-10-11 Sun 17:48

Validate