yGOT part 1
Code for this will be leveraged within the git repo
Intro
In this part 1 we will take a elaborate look into yGOT. I recently came back from a conference where Openconfig and yang were talked aboout a lot but I had the feeling that neither were something people were using a good amount of but where generally extremely curious about. yGOT for me is really interesting. I have been talking about such a thing over the past few years.
If I were to compare yGOT and this general sense of feeling you would walk away with is more or less something similar to the way pulumi operates. At its core it is infrastructure as software. You are taking a well defined YANG schema making a data model around it modeled within Go structs which then allows a user to reference the go structs to create JSON which can be in turned used to configure the targeted device. This is a complicated topic so I will do my best to break it all down within this post.
If openconfig and Yang are generally new to you I would suggest watching a few videos where two of my colleagues and I got together to discuss all things YANG, Openconfig and gNMI.
The entire playlist can be found here
In part 1 I plan to make a example of how to use yGOT to create go structs for everything within the system openconfig yang model.
If you do not feel like reading this short novel this is the end result of what ygot does all within a gif.
In Part 2 I plan on configuring a device with this json data.
What is yGOT?
yGOT allows for someone to take yang schema for example the openconfig yang model for system and create go based typed value code out of it. So that a person can directly write code against a device. This is extremely attractive if you are a avid gopher or anyone who is embracing a lot of network programmability.
A visual for how this is constructed.
I mean sounds pretty simple right? I say this sarcastically. This is not a simple thing. It is rather fascinating. Since the YANG schema tells us all the types that the device has to conform to ie a vlan must be a integer or a ntp server has to be a type of oc-inet:host which is simply a ipv4 address. Yes, there are YANG specific values. yGOT is able to take this schema and build go structs to then create JSON.
This JSON is then used against the gNMI interface in the form of what is called a SET request to configure the device.
Generating the yGOT go structs.
To generate the yGOT go structs we have two options. Either generate it out of yGOT go module or we can use the binary from the ygot repo. In this blog post we will demonstrate using the ygot binary.
Build the binary
git clone https://github.com/openconfig/ygot
cd ygot/generator
go build
This will build the generator binary. This is what transforms the yang schema to go structs.
Yang Files
cd yang
yang ls
openconfig-aaa-radius.yang openconfig-alarms.yang openconfig-license.yang openconfig-procmon.yang openconfig-system.yang
openconfig-aaa-tacacs.yang openconfig-alarm-types.yang openconfig-messages.yang openconfig-system-logging.yang openconfig-types.yang
openconfig-aaa-types.yang openconfig-extensions.yang openconfig-platform-types.yang openconfig-system-management.yang openconfig-yang-types.yang
openconfig-aaa.yang openconfig-inet-types.yang openconfig-platform.yang openconfig-system-terminal.yang
Woah that is a lot of files! So the way this works with yGOT is that it needs access to all yang files. So the YANG file I really want to leverage is the openconfig-system.yang model. This yang model for example imports other YANG models. So they are necessary. For example,
openconfig-system.yang
module openconfig-system {
yang-version "1";
// namespace
namespace "http://openconfig.net/yang/system";
prefix "oc-sys";
// import some basic types
import openconfig-inet-types { prefix oc-inet; }
import openconfig-yang-types { prefix oc-yang; }
import openconfig-types { prefix oc-types; }
import openconfig-extensions { prefix oc-ext; }
import openconfig-aaa { prefix oc-aaa; }
import openconfig-system-logging { prefix oc-log; }
import openconfig-system-terminal { prefix oc-sys-term; }
import openconfig-procmon { prefix oc-proc; }
import openconfig-platform { prefix oc-platform; }
import openconfig-alarms { prefix oc-alarms; }
import openconfig-messages { prefix oc-messages; }
import openconfig-license { prefix oc-license; }
import openconfig-network-instance { prefix oc-ni; }
So we need all of those imports and for example openconfig-inet-types might import other yang files so we need those as well.
So we have our yGOT generator binary and our yang files within the repo currently. We are now ready to generator some go code!
➜ ygotblogpost ls -l
total 14324
-rwxrwxr-x 1 burnyd burnyd 14660522 Nov 18 20:27 generator
drwxrwxr-x 2 burnyd burnyd 4096 Nov 18 20:28 yang
Generating the go structs
We need to now take the YANG schema files located within the /yang directory and turn them into go structs. We do this by using the generator binary we had previously. This will create a go package.
# Create a directory for the go pkg
mkdir -p pkg/ocsystem/
# Run the ygot generator
./generator -output_file=pkg/ocsystem/ocsystem.go \
-package_name=ocsystem -path=yang/ -generate_fakeroot \
-fakeroot_name=device -exclude_modules=ietf-interfaces \
-compress_paths=true \
yang/openconfig-system.yang
Now lets see what happens here within the pkg/ocsystem directory
➜ ygotblogpost ls -l pkg/ocsystem/ocsystem.go
-rw-rw-r-- 1 burnyd burnyd 390977 Nov 19 15:12 pkg/ocsystem/ocsystem.go
➜ ygotblogpost wc -l pkg/ocsystem/ocsystem.go
6561 pkg/ocsystem/ocsystem.go
So we see that ocsystem.go was created. Lets breakdown some of the important flags within the generator and what they are.
-output_file=pkg/ocsystem/ocsystem.go - Tells yGOT where to output the go library.
-package_name=ocsystem - Tells yGOT what to call the go package. Inside of ocsystem.go it will be called ocsystem.
-path=yang/ - The path in which the yang files are located.
-generate_fakeroot and -fakeroot_name=device - Are the root of the structs. More on this later. But device is the top level struct and every method and other struct are connect to it.
yang/openconfig-system.yang - This is the yang model in which we are targeting.
A dive into ocsystem.go
// Device represents the /device YANG schema element.
type Device struct {
Component map[string]*Component `path:"components/component" module:"openconfig-platform/openconfig-platform"`
Messages *Messages `path:"messages" module:"openconfig-messages"`
System *System `path:"system" module:"openconfig-system"`
}
This is particularly interesting. yGOT created this top level struct that also has pointers to types within it Component, Messages and System. Which are all their own types. Typically where we would see something like json tags or yaml tags there are tags to the specific path. Lets take a look at the System type.
// System represents the /openconfig-system/system YANG schema element.
type System struct {
Aaa *System_Aaa `path:"aaa" module:"openconfig-system"`
Alarm map[string]*System_Alarm `path:"alarms/alarm" module:"openconfig-system/openconfig-system"`
BootTime *uint64 `path:"state/boot-time" module:"openconfig-system/openconfig-system"`
Clock *System_Clock `path:"clock" module:"openconfig-system"`
Cpu map[System_Cpu_Index_Union]*System_Cpu `path:"cpus/cpu" module:"openconfig-system/openconfig-system"`
CurrentDatetime *string `path:"state/current-datetime" module:"openconfig-system/openconfig-system"`
Dns *System_Dns `path:"dns" module:"openconfig-system"`
DomainName *string `path:"config/domain-name" module:"openconfig-system/openconfig-system"`
GrpcServer *System_GrpcServer `path:"grpc-server" module:"openconfig-system"`
Hostname *string `path:"config/hostname" module:"openconfig-system/openconfig-system"`
License *System_License `path:"license" module:"openconfig-system"`
Logging *System_Logging `path:"logging" module:"openconfig-system"`
LoginBanner *string `path:"config/login-banner" module:"openconfig-system/openconfig-system"`
Memory *System_Memory `path:"memory" module:"openconfig-system"`
Messages *System_Messages `path:"messages" module:"openconfig-system"`
MotdBanner *string `path:"config/motd-banner" module:"openconfig-system/openconfig-system"`
Ntp *System_Ntp `path:"ntp" module:"openconfig-system"`
Process map[uint64]*System_Process `path:"processes/process" module:"openconfig-system/openconfig-system"`
SshServer *System_SshServer `path:"ssh-server" module:"openconfig-system"`
TelnetServer *System_TelnetServer `path:"telnet-server" module:"openconfig-system"`
}
Look at all those types! This is great. Probably the easiest of easiest examples would be having another package call this out and simply creating a hostname. Here is an example I wrote for out open management site.
Lets take NTP for example. We would want a good way of rendering a NTP related JSON file to configure a NTP server.
NTP uses the *System_Ntp type
// System_Ntp represents the /openconfig-system/system/ntp YANG schema element.
type System_Ntp struct {
AuthMismatch *uint64 `path:"state/auth-mismatch" module:"openconfig-system/openconfig-system"`
EnableNtpAuth *bool `path:"config/enable-ntp-auth" module:"openconfig-system/openconfig-system"`
Enabled *bool `path:"config/enabled" module:"openconfig-system/openconfig-system"`
NtpKey map[uint16]*System_Ntp_NtpKey `path:"ntp-keys/ntp-key" module:"openconfig-system/openconfig-system"`
NtpSourceAddress *string `path:"config/ntp-source-address" module:"openconfig-system/openconfig-system"`
Server map[string]*System_Ntp_Server `path:"servers/server" module:"openconfig-system/openconfig-system"`
}
The server key is a map[string]*System_Ntp_Server
// System_Ntp_Server represents the /openconfig-system/system/ntp/servers/server YANG schema element.
type System_Ntp_Server struct {
Address *string `path:"config/address|address" module:"openconfig-system/openconfig-system|openconfig-system"`
AssociationType E_OpenconfigSystem_Server_AssociationType `path:"config/association-type" module:"openconfig-system/openconfig-system"`
Iburst *bool `path:"config/iburst" module:"openconfig-system/openconfig-system"`
Offset *uint64 `path:"state/offset" module:"openconfig-system/openconfig-system"`
PollInterval *uint32 `path:"state/poll-interval" module:"openconfig-system/openconfig-system"`
Port *uint16 `path:"config/port" module:"openconfig-system/openconfig-system"`
Prefer *bool `path:"config/prefer" module:"openconfig-system/openconfig-system"`
RootDelay *uint32 `path:"state/root-delay" module:"openconfig-system/openconfig-system"`
RootDispersion *uint64 `path:"state/root-dispersion" module:"openconfig-system/openconfig-system"`
Stratum *uint8 `path:"state/stratum" module:"openconfig-system/openconfig-system"`
Version *uint8 `path:"config/version" module:"openconfig-system/openconfig-system"`
}
So if this all aligns for you. We would have typed access to create a NTP server and all the keys that are necessary for NTP servers. ie Address, Port etc that are within the System_Ntp_Server.
Lets write some code.
go mod init github.com/burnyd/ygotblogpost
touch main.go
main.go
Lets take a look at the main.go file where we are going to leverage all of the go structs that are relevent to create json for the device.
package main
import (
"fmt"
"github.com/burnyd/ygotblogpost/pkg/ocsystem"
"github.com/openconfig/ygot/ygot"
)
func main() {
// Create a type for ntpserver. Typically, we would pass this into a function just for demo. Give it an IPv4 address.
var ntpserver *string
ntpserverIp := "1.2.3.4"
ntpserver = &ntpserverIp
// Create a NtpServer which is of type System_Ntp_Server and pass in the 1.2.3.4 address.
NtpServer := ocsystem.System_Ntp_Server{
Address: ygot.String(*ntpserver),
}
// Cerate a Map of System_Ntp_Server
NtpMap := make(map[string]*ocsystem.System_Ntp_Server)
// Make one of the keys the ntp server address we already created.
NtpMap[*ntpserver] = &NtpServer
NtpSys := ocsystem.System_Ntp{
Server: NtpMap,
}
// Create a pointer to System{} struct and pass into the NTP field the NTP data.
Sys := &ocsystem.System{Ntp: &NtpSys}
// Marshall the JSON for Sys
json, err := ygot.EmitJSON(Sys, &ygot.EmitJSONConfig{
Format: ygot.RFC7951,
Indent: " ",
RFC7951Config: &ygot.RFC7951JSONConfig{
AppendModuleName: true,
},
})
if err != nil {
panic(fmt.Sprintf("Value error: %v", err))
}
// Print the json
fmt.Println(json)
}
var ntpserver *string
ntpserverIp := "1.2.3.4"
ntpserver = &ntpserverIp
This is really so that we can initialize a ip address of a server. Everything is generally a pointer within ygot.
NtpServer := ocsystem.System_Ntp_Server{
Address: ygot.String(*ntpserver),
}
NtpServer is the type of System_Ntp_Server which maps to system-ntp-server-config mapping within yang as follows
grouping system-ntp-server-config {
description
"Configuration data for NTP servers";
leaf address {
type oc-inet:host;
description
"The address or hostname of the NTP server.";
}
leaf port {
type oc-inet:port-number;
default 123;
description
"The port number of the NTP server.";
}
So at this point we have our ntp server address inserted. Now to satisfy the type we need to create a map of map[string]*ocsystem.System_Ntp_Server because that is what the System_Ntp struct needs. For example,
System_Ntp struct
type System_Ntp struct {
AuthMismatch *uint64 `path:"state/auth-mismatch" module:"openconfig-system/openconfig-system"`
EnableNtpAuth *bool `path:"config/enable-ntp-auth" module:"openconfig-system/openconfig-system"`
Enabled *bool `path:"config/enabled" module:"openconfig-system/openconfig-system"`
NtpKey map[uint16]*System_Ntp_NtpKey `path:"ntp-keys/ntp-key" module:"openconfig-system/openconfig-system"`
NtpSourceAddress *string `path:"config/ntp-source-address" module:"openconfig-system/openconfig-system"`
Server map[string]*System_Ntp_Server `path:"servers/server" module:"openconfig-system/openconfig-system"`
}
NtpMap := make(map[string]*ocsystem.System_Ntp_Server)
NtpMap[*ntpserver] = &NtpServer
NtpSys := ocsystem.System_Ntp{
Server: NtpMap,
}
Sys := &ocsystem.System{Ntp: &NtpSys}
We make NtpMap key the 1.2.3.4 ip address. We then take the System_Ntp struct and simply fill in the map.
Sys := &ocsystem.System{Ntp: &NtpSys} is where to tie everything into the System struct. We could potentially configure other things like syslog, hostname etc if we wanted to.
The rest of the code simply marshalls this and allows for someone to print it out to the user.
Here is a gif of writing out the code. You can see here all the typed nature which allows you to see all the values. In my opinion this is the most powerful portion.
Result
go run main.go
{
"openconfig-system:ntp": {
"servers": {
"server": [
{
"address": "1.2.3.4",
"config": {
"address": "1.2.3.4"
}
}
]
}
}
}