Compiler and Virtual Machine </a></li><li><a href="/topics/templates2.html" class="sidebar-link">Template Language </a></li><li><a href="/topics/vm.html" aria-current="page" class="active sidebar-link">Compiler and Virtual Machine </a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/topics/vm.html#source-code-storage-and-compilation" class="sidebar-link">Source code storage and compilation </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#virtual-machine-structures" class="sidebar-link">Virtual machine structures </a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/topics/vm.html#vm-structure" class="sidebar-link">VM Structure </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#block-structure" class="sidebar-link">Block structure </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#objinfo-structure" class="sidebar-link">ObjInfo structure </a></li></ul></li><li class="sidebar-sub-header"><a href="/topics/vm.html#virtual-machine-commands" class="sidebar-link">Virtual machine commands </a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/topics/vm.html#bytecode-structure" class="sidebar-link">ByteCode structure </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#command-identifiers" class="sidebar-link">Command identifiers </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#stack-operation-commands" class="sidebar-link">Stack operation commands </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#runtime-structure" class="sidebar-link">Runtime structure </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#runcode-function" class="sidebar-link">RunCode function </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#other-functions-for-operations-with-vm" class="sidebar-link">Other functions for operations with VM </a></li></ul></li><li class="sidebar-sub-header"><a href="/topics/vm.html#compiler" class="sidebar-link">Compiler </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#lexical-analyzer" class="sidebar-link">Lexical analyzer </a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/topics/vm.html#lextable-lextable-go" class="sidebar-link">lextable/lextable.go </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#lex-go" class="sidebar-link">lex-go </a></li></ul></li><li class="sidebar-sub-header"><a href="/topics/vm.html#needle-language" class="sidebar-link">Needle language </a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/topics/vm.html#lexemes" class="sidebar-link">Lexemes </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#types" class="sidebar-link">Types </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#expressions" class="sidebar-link">Expressions </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#scope" class="sidebar-link">Scope </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#contract-execution" class="sidebar-link">Contract execution </a></li><li class="sidebar-sub-header"><a href="/topics/vm.html#backus-naur-form-bnf" class="sidebar-link">Backus–Naur Form (BNF) </a></li></ul></li></ul></li><li><a href="/topics/daemons.html" class="sidebar-link">Daemon </a></li></ul></section></li></ul> </aside> <main class="page"> <div class="theme-default-content content__default"><h1 id="compiler-and-virtual-machine"><a href="#compiler-and-virtual-machine" class="header-anchor">#</a> Compiler and Virtual Machine </h1> <ul><li><a href="#source-code-storage-and-compilation">Source code storage and compilation</a></li> <li><a href="#virtual-machine-structures">Virtual machine structures</a> <ul><li><a href="#vm-structure">VM Structure</a></li> <li><a href="#block-structure">Block structure</a></li> <li><a href="#objinfo-structure">ObjInfo structure</a> <ul><li><a href="#contractinfo-structure">ContractInfo structure</a></li> <li><a href="#fieldinfo-structure">FieldInfo structure</a></li> <li><a href="#funcinfo-structure">FuncInfo structure</a></li> <li><a href="#funcname-structure">FuncName Structure</a></li> <li><a href="#extfuncinfo-structure">ExtFuncInfo structure</a></li> <li><a href="#varinfo-structure">VarInfo structure</a></li> <li><a href="#objextend-value">ObjExtend value</a></li></ul></li></ul></li> <li><a href="#virtual-machine-commands">Virtual machine commands</a> <ul><li><a href="#bytecode-structure">ByteCode structure</a></li> <li><a href="#command-identifiers">Command identifiers</a></li> <li><a href="#stack-operation-commands">Stack operation commands</a></li> <li><a href="#runtime-structure">Runtime structure</a> <ul><li><a href="#blockstack-structure">blockStack structure</a></li></ul></li> <li><a href="#runcode-function">RunCode function</a></li> <li><a href="#other-functions-for-operations-with-vm">Other functions for operations with VM</a></li></ul></li> <li><a href="#compiler">Compiler</a></li> <li><a href="#lexical-analyzer">Lexical analyzer</a> <ul><li><a href="#lextable-lextable-go">lextable/lextable.go</a></li> <li><a href="#lex-go">lex.go</a></li></ul></li> <li><a href="#needle-language">Needle language</a> <ul><li><a href="#lexemes">Lexemes</a></li> <li><a href="#types">Types</a></li> <li><a href="#expressions">Expressions</a></li> <li><a href="#scope">Scope</a></li> <li><a href="#contract-execution">Contract execution</a></li> <li><a href="#backus-naur-form-bnf">Backus–Naur Form (BNF)</a></li></ul></li></ul> <p>This section involves program compilation and Needle language operations in the Virtual Machine (VM).</p> <h2 id="source-code-storage-and-compilation"><a href="#source-code-storage-and-compilation" class="header-anchor">#</a> Source code storage and compilation </h2> <p>Contracts and functions are written with Golang and stored in the contract tables of ecosystems.</p> <p>When a contract is executed, its source code will be read from the database and compiled into bytecode.</p> <p>When a contract is changed, its source code will be updated and saved in the database. Then, the source code is compiled, thereby updating the bytecode in the corresponding virtual machine.</p> <p>As bytecodes are not physically saved, it will be compiled anew when the program is executed again.</p> <p>The entire source code described in the contract table of each ecosystem is compiled into a virtual machine in strict order, and the status of the virtual machine is the same on all nodes.</p> <p>When the contract is called, the virtual machine will not change its status in any way. The execution of any contract or calling of any function occurs on a separate running stack created during each external call.</p> <p>Each ecosystem can have a so-called virtual ecosystem, which can be used within a node in conjunction with tables outside the blockchain, without direct affection on the blockchain or other virtual ecosystems. In this case, the node hosting such a virtual ecosystem will compile its contract and create its own virtual machine.</p> <h2 id="virtual-machine-structures"><a href="#virtual-machine-structures" class="header-anchor">#</a> Virtual machine structures </h2> <h3 id="vm-structure"><a href="#vm-structure" class="header-anchor">#</a> VM Structure </h3> <p>A virtual machine is organized in memory as a structure like below.</p> <div class="language- extra-class"><pre class="language-text"><code>type VM struct { Block ExtCost func(string) int64 FuncCallsDB map[string]struct{} Extern bool ShiftContract int64 logger *log.Entry } </code></pre></div><p>A VM structure has the following elements:</p> <ul><li>Block - contains a <a href="#block-structure">block structure</a>;</li> <li>ExtCost - a function returns the cost of executing an external golang function;</li> <li>FuncCallsDB - a collection of Golang function names. This function returns the execution cost as the first parameter. These functions use EXPLAIN to calculate the cost of database processing;</li> <li>Extern - a Boolean flag indicating whether a contract is an external contract. It is set to true when a VM is created. Contracts called are not displayed when the code is compiled. In other words, it allows to call the contract code determined in the future;</li> <li>ShiftContract - ID of the first contract in the VM;</li> <li>logger - VM error log output.</li></ul> <h3 id="block-structure"><a href="#block-structure" class="header-anchor">#</a> Block structure </h3> <p>A virtual machine is a tree composed of <strong>Block type</strong> objects.</p> <p>A block is an independent unit that contains some bytecodes. In simple terms, everything you put in the braces (<code>{}</code>) in the language is a block.</p> <p>For example, the following code would create a block with functions. This block also contains another block with an if statement, which contains a block with a while statement.</p> <div class="language- extra-class"><pre class="language-text"><code>func my() { if true { while false { ... } } } </code></pre></div><p>The block is organized in the memory as a structure like below.</p> <div class="language- extra-class"><pre class="language-text"><code>type Block struct { Objects map[string]*ObjInfo Type int Owner *OwnerInfo Info interface{} Parent *Block Vars []reflect.Type Code ByteCodes Children Blocks } </code></pre></div><p>A block structure consists of the following elements:</p> <ul><li><strong>Objects</strong> - a map of internal objects of the pointer type <a href="#objinfo-structure">ObjInfo</a>. For example, if there is a variable in the block, you can get information about it by its name;</li> <li><strong>Type</strong> - the type of the block. For a function block, its type is <strong>ObjFunc</strong>; for a contract block, its type is <strong>ObjContract</strong>;</li> <li><strong>Owner</strong> - a structure of <strong>OwnerInfo</strong> pointer type. This structure contains information about the owner of the compiled contract, which is specified during contract compilation or obtained from the <strong>contracts</strong> table;</li> <li><strong>Info</strong> - it contains information about the object, which depends on the block type;</li> <li><strong>Parent</strong> - a pointer to the parent block;</li> <li><strong>Vars</strong> - an array containing the types of current block variables;</li> <li><strong>Code</strong> - the bytecode of the block itself, which will be executed when the control rights are passed to the block, for example, function calls or loop bodies;</li> <li><strong>Children</strong> - an array containing sub-blocks, such as function nesting, loops, conditional operators.</li></ul> <h3 id="objinfo-structure"><a href="#objinfo-structure" class="header-anchor">#</a> ObjInfo structure </h3> <p>The ObjInfo structure contains information about internal objects.</p> <div class="language- extra-class"><pre class="language-text"><code>type ObjInfo struct { Type int Value interface{} } </code></pre></div><p>The ObjInfo structure has the following elements:</p> <ul><li><p><strong>Type</strong> is the object type, which has any of the following values:</p> <ul><li><strong>ObjContract</strong> – <a href="#contractinfo-structure">contract</a>;</li> <li><strong>ObjFunc</strong> - function;</li> <li><strong>ObjExtFunc</strong> - external golang function;</li> <li><strong>ObjVar</strong> - variable;</li> <li><strong>ObjExtend</strong> - $name variable.</li></ul></li> <li><p><strong>Value</strong> – it contains the structure of each type.</p></li></ul> <h4 id="contractinfo-structure"><a href="#contractinfo-structure" class="header-anchor">#</a> ContractInfo structure </h4> <p>Pointing to the <strong>ObjContract</strong> type, and the <strong>Value</strong> field contains a <strong>ContractInfo</strong> structure.</p> <div class="language- extra-class"><pre class="language-text"><code>type ContractInfo struct { ID uint32 Name string Owner *OwnerInfo Used map[string]bool Tx *[]*FieldInfo } </code></pre></div><p>The ContractInfo structure has the following elements:</p> <ul><li><strong>ID</strong> - contract ID, displayed in the blockchain when calling the contract;</li> <li><strong>Name</strong> - contract name;</li> <li><strong>Owner</strong> - other information about the contract;</li> <li><strong>Used</strong> - map of contracts names that has been called;</li> <li><strong>Tx</strong> - a data array described in the <a href="/topics/script.html#data-section">data section</a> of the contract.</li></ul> <h4 id="fieldinfo-structure"><a href="#fieldinfo-structure" class="header-anchor">#</a> FieldInfo structure </h4> <p>The FieldInfo structure is used in the <strong>ContractInfo</strong> structure and describes elements in <a href="/topics/script.html#data-section">data section</a> of a contract.</p> <div class="language- extra-class"><pre class="language-text"><code>type FieldInfo struct { Name string Type reflect.Type Original uint32 Tags string } </code></pre></div><p>The FieldInfo structure has the following elements:</p> <ul><li><strong>Name</strong> - field name;</li> <li><strong>Type</strong> - field type;</li> <li><strong>Original</strong> - optional field;</li> <li><strong>Tags</strong> - additional labels for this field.</li></ul> <h4 id="funcinfo-structure"><a href="#funcinfo-structure" class="header-anchor">#</a> FuncInfo structure </h4> <p>Pointing to the ObjFunc type, and the Value field contains a FuncInfo structure.</p> <div class="language- extra-class"><pre class="language-text"><code>type FuncInfo struct { Params []reflect.Type Results []reflect.Type Names *map[string]FuncName Variadic bool ID uint32 } </code></pre></div><p>The FuncInfo structure has the following elements:</p> <ul><li><strong>Params</strong> - an array of parameter types;</li> <li><strong>Results</strong> - an array of returned types;</li> <li><strong>Names</strong> - map of data for tail functions, for example, <code>DBFind().Columns ()</code>;</li> <li><strong>Variadic</strong> - true if the function can have a variable number of parameters;</li> <li><strong>ID</strong> - function ID.</li></ul> <h4 id="funcname-structure"><a href="#funcname-structure" class="header-anchor">#</a> FuncName Structure </h4> <p>The FuncName structure is used for FuncInfo and describes the data of a tail function.</p> <div class="language- extra-class"><pre class="language-text"><code>type FuncName struct { Params []reflect.Type Offset []int Variadic bool } </code></pre></div><p>The FuncName structure has the following elements:</p> <ul><li><strong>Params</strong> - an array of parameter types;</li> <li><strong>Offset</strong> - the array of offsets for these variables. In fact, the values of all parameters in a function can be initialized with the dot .;</li> <li><strong>Variadic</strong> - true if the tail function can have a variable number of parameters.</li></ul> <h4 id="extfuncinfo-structure"><a href="#extfuncinfo-structure" class="header-anchor">#</a> ExtFuncInfo structure </h4> <p>Pointing to the ObjExtFunc type, and the Value field contains a ExtFuncInfo structure. It is used to describe golang functions.</p> <div class="language- extra-class"><pre class="language-text"><code>type ExtFuncInfo struct { Name string Params []reflect.Type Results []reflect.Type Auto []string Variadic bool Func interface{} } </code></pre></div><p>The ExtFuncInfo structure has the following elements:</p> <ul><li><strong>Name</strong>, <strong>Params</strong>, <strong>Results</strong> parameters have the same structure as <a href="#funcinfo-structure">FuncInfo</a>;</li> <li><strong>Auto</strong> - an array of variables. If any, passes to the function as an additional parameter. For example, a variable of type SmartContract sc;</li> <li><strong>Func</strong> - golang functions.</li></ul> <h4 id="varinfo-structure"><a href="#varinfo-structure" class="header-anchor">#</a> VarInfo structure </h4> <p>Pointing to the <strong>ObjVar</strong> type, and the <strong>Value</strong> field contains a <strong>VarInfo</strong> structure.</p> <div class="language- extra-class"><pre class="language-text"><code>type VarInfo struct { Obj *ObjInfo Owner *Block } </code></pre></div><p>The VarInfo structure has the following elements:</p> <ul><li><strong>Obj</strong> - information about the type and value of the variable;</li> <li><strong>Owner</strong> - Pointer to the owner block.</li></ul> <h4 id="objextend-value"><a href="#objextend-value" class="header-anchor">#</a> ObjExtend value </h4> <p>Pointing to the <strong>ObjExtend</strong> type, and the <strong>Value</strong> field contains a string containing the name of the variable or function.</p> <h2 id="virtual-machine-commands"><a href="#virtual-machine-commands" class="header-anchor">#</a> Virtual machine commands </h2> <h3 id="bytecode-structure"><a href="#bytecode-structure" class="header-anchor">#</a> ByteCode structure </h3> <p>A bytecode is a sequence of <strong>ByteCode</strong> type structures.</p> <div class="language- extra-class"><pre class="language-text"><code>type ByteCode struct { Cmd uint16 Value interface{} } </code></pre></div><p>This structure has the following fields:</p> <ul><li><strong>Cmd</strong> - the identifier of the storage commands;</li> <li><strong>Value</strong> - contains the operand (value).</li></ul> <p>In general, commands perform an operation on the top element of the stack and writes the result value into it if necessary.</p> <h3 id="command-identifiers"><a href="#command-identifiers" class="header-anchor">#</a> Command identifiers </h3> <p>Identifiers of the virtual machine commands are described in the vm/cmds_list.go file.</p> <ul><li><strong>cmdPush</strong> – put a value from the Value field to the stack. For example, put numbers and lines to the stack;</li> <li><strong>cmdVar</strong> - put the value of a variable to the stack. Value contains a pointer to the VarInfo structure and information about the variable;</li> <li><strong>cmdExtend</strong> – put the value of an external variable to the stack. Value contains a string with the variable name (starting with $);</li> <li><strong>cmdCallExtend</strong> – call an external function (starting with <code>$</code>). The parameters of the function are obtained from the stack, and the results are placed to the stack. Value contains a function name (starting with <code>$</code>);</li> <li><strong>cmdPushStr</strong> – put the string in Value to the stack;</li> <li><strong>cmdCall</strong> - calls the virtual machine function. Value contains a <strong>ObjInfo</strong> structure. This command is applicable to the <strong>ObjExtFunc</strong> golang function and <strong>ObjFunc</strong> Needle function. If a function is called, its parameters will be obtained from the stack and the result values will be placed to the stack;</li> <li><strong>cmdCallVari</strong> - similar to the <strong>cmdCall</strong> command, it calls the virtual machine function. This command is used to call a function with a variable number of parameters;</li> <li><strong>cmdReturn</strong> - used to exit the function. The return values will be put to the stack, and the Value field is not used;</li> <li><strong>cmdIf</strong> – transfer control to the bytecode in the <strong>block</strong> structure, which is passed in the Value field. The control will be transferred to the stack only when the top element of the stack is called by the <em>valueToBool</em> function and returned <code>true</code>. Otherwise, the control will be transferred to the next command;</li> <li><strong>cmdElse</strong> - this command works in the same way as the <strong>cmdIf</strong>, but only when the top element of the stack is called by the valueToBool function and returned <code>false</code>, the control will be transferred to the specified block;</li> <li><strong>cmdAssignVar</strong> – get a list of variables of type <strong>VarInfo</strong> from Value. These variables use the <strong>cmdAssign</strong> command to get the value;</li> <li><strong>cmdAssign</strong> – assign the value in the stack to the variable obtained by the <strong>cmdAssignVar</strong> command;</li> <li><strong>cmdLabel</strong> - defines a label when control is returned during the while loop;</li> <li><strong>cmdContinue</strong> - this command transfers control to the <strong>cmdLabel</strong> label. When executing a new iteration of the loop, Value is not used;</li> <li><strong>cmdWhile</strong> – use valueToBool to check the top element of the stack. If this value is <code>true</code>, the <strong>block</strong> structure will be called from the value field;</li> <li><strong>cmdBreak</strong> - exits the loop;</li> <li><strong>cmdIndex</strong> – put the value in map or array into the stack by index, without using Value. For example, <code>(map | array) (index value) => (map | array [index value])</code>;</li> <li><strong>cmdSetIndex</strong> – assigns the value of the top element of the stack to elements of map or array, without using Value. For example, <code>(map | array) (index value) (value) => (map | array)</code>;</li> <li><strong>cmdFuncName</strong> - adds parameters that are passed using sequential descriptions divided by dot . For example, <code>func name => Func (...) .Name (...)</code>;</li> <li><strong>cmdUnwrapArr</strong> - defines a Boolean flag if the top element of the stack is an array;</li> <li><strong>cmdMapInit</strong> – initializes the value of map;</li> <li><strong>cmdArrayInit</strong> – initializes the value of array;</li> <li><strong>cmdError</strong> - this command is created when a contract or function terminates with a specified <code>error, warning, info</code>.</li></ul> <h3 id="stack-operation-commands"><a href="#stack-operation-commands" class="header-anchor">#</a> Stack operation commands </h3> <blockquote><p>Note</p></blockquote> <blockquote><p>In the current version, automatic type conversion is not fully applicable for these commands. For example,</p></blockquote> <blockquote><p><code>string + float | int | decimal => float | int | decimal, float + int | str => float, but int + string => runtime error</code>.</p></blockquote> <p>The following are commands for direct stack processing. The Value field is not used in these commands.</p> <ul><li><strong>cmdNot</strong> - logical negation. <code>(val) => (!ValueToBool(val))</code>;</li> <li><strong>cmdSign</strong> - change of sign. <code>(val) => (-val)</code>;</li> <li><strong>cmdAdd</strong> - addition. <code>(val1)(val2) => (val1 + val2)</code>;</li> <li><strong>cmdSub</strong> - subtraction. <code>(val1)(val2) => (val1-val2)</code>;</li> <li><strong>cmdMul</strong> - multiplication. <code>(val1)(val2) => (val1 * val2)</code>;</li> <li><strong>cmdDiv</strong> - division. <code>(val1)(val2) => (val1 / val2)</code>;</li> <li><strong>cmdAnd</strong> - logical AND. <code>(val1)(val2) => (valueToBool(val1) && valueToBool(val2))</code>;</li> <li><strong>cmdOr</strong> - logical OR. <code>(val1)(val2) => (valueToBool(val1) || valueToBool(val2))</code>;</li> <li><strong>cmdEqual</strong> - equality comparison, bool is returned. <code>(val1)(val2) => (val1 == val2)</code>;</li> <li><strong>cmdNotEq</strong> - inequality comparison, bool is returned. <code>(val1)(val2) => (val1 != val2)</code>;</li> <li><strong>cmdLess</strong> - less-than comparison, bool is returned. <code>(val1)(val2) => (val1 <val2)</code>;</li> <li><strong>cmdNotLess</strong> - greater-than-or-equal comparison, bool is returned. <code>(val1)(val2) => (val1 >= val2)</code>;</li> <li><strong>cmdGreat</strong> - greater-than comparison, bool is returned. <code>(val1)(val2) => (val1> val2)</code>;</li> <li><strong>cmdNotGreat</strong> - less-than-or-equal comparison, bool is returned. <code>(val1)(val2) => (val1 <= val2)</code>.</li></ul> <h3 id="runtime-structure"><a href="#runtime-structure" class="header-anchor">#</a> Runtime structure </h3> <p>The execution of bytecodes will not affect the virtual machine. For example, it allows various functions and contracts to run simultaneously in a single virtual machine. The Runtime structure is used to run functions and contracts, as well as any expressions and bytecode.</p> <div class="language- extra-class"><pre class="language-text"><code>type RunTime struct { stack []interface{} blocks []*blockStack vars []interface{} extend *map[string]interface{} vm *VM cost int64 err error } </code></pre></div><ul><li><strong>stack</strong> - the stack to execute the bytecode;</li> <li><strong>blocks</strong> - block calls stack;</li> <li><strong>vars</strong> - stack of variables. Its variable will be added to the stack of variables when the bytecode is called in the block. After exiting the block, the size of the stack of variables will return to the previous value;</li> <li><strong>extend</strong> - a pointer to map with values of external variables (<code>$name</code>);</li> <li><strong>vm</strong> - a virtual machine pointer;</li> <li><strong>cost</strong> - fuel unit of the resulting cost of execution;</li> <li><strong>err</strong> - error occurred during execution.</li></ul> <h4 id="blockstack-structure"><a href="#blockstack-structure" class="header-anchor">#</a> blockStack structure </h4> <p>The blockStack structure is used in the Runtime structure.</p> <div class="language- extra-class"><pre class="language-text"><code>type blockStack struct { Block *Block Offset int } </code></pre></div><ul><li><strong>Block</strong> - a pointer to the block being executed;</li> <li><strong>Offset</strong> – the offset of the last command executed in the bytecode of the specified block.</li></ul> <h3 id="runcode-function"><a href="#runcode-function" class="header-anchor">#</a> RunCode function </h3> <p>Bytecodes are executed in the <strong>RunCode</strong> function. It contains a loop that performs the corresponding operation for each bytecode command. Before processing a bytecode, the data required must be initialized.</p> <p>New blocks are added to other blocks.</p> <div class="language- extra-class"><pre class="language-text"><code>rt.blocks = append(rt.blocks, &blockStack{block, len(rt.vars)}) </code></pre></div><p>Next, get the information of relevant parameters of the tail function. These parameters are contained in the last element of the stack.</p> <div class="language- extra-class"><pre class="language-text"><code>var namemap map[string][]interface{} if block.Type == ObjFunc && block.Info.(*FuncInfo).Names != nil { if rt.stack[len(rt.stack)-1] != nil { namemap = rt.stack[len(rt.stack)-1].(map[string][]interface{}) } rt.stack = rt.stack[:len(rt.stack)-1] } </code></pre></div><p>Then, all variables defined in the current block must be initialized with their initial values.</p> <div class="language- extra-class"><pre class="language-text"><code>start := len(rt.stack) varoff := len(rt.vars) for vkey, vpar := range block.Vars { rt.cost-- var value interface{} </code></pre></div><p>Since variables in the function are also variables, we need to retrieve them from the last element of the stack in the order described by the function itself.</p> <div class="language- extra-class"><pre class="language-text"><code> if block.Type == ObjFunc && vkey <len(block.Info.(*FuncInfo).Params) { value = rt.stack[start-len(block.Info.(*FuncInfo).Params)+vkey] } else { </code></pre></div><p>Initialize local variables with their initial values.</p> <div class="language- extra-class"><pre class="language-text"><code> value = reflect.New(vpar).Elem().Interface() if vpar == reflect.TypeOf(map[string]interface{}{}) { value = make(map[string]interface{}) } else if vpar == reflect.TypeOf([]interface{}{}) { value = make([]interface{}, 0, len(rt.vars)+1) } } rt.vars = append(rt.vars, value) } </code></pre></div><p>Next, update the values of variable parameters passed in the tail function.</p> <div class="language- extra-class"><pre class="language-text"><code>if namemap != nil { for key, item := range namemap { params := (*block.Info.(*FuncInfo).Names)[key] for i, value := range item { if params.Variadic && i >= len(params.Params)-1 { </code></pre></div><p>If variable parameters passed belongs to a variable number of parameters, then these parameters will be combined into an array of variables.</p> <div class="language- extra-class"><pre class="language-text"><code> off := varoff + params.Offset[len(params.Params)-1] rt.vars[off] = append(rt.vars[off].([]interface{}), value) } else { rt.vars[varoff+params.Offset[i]] = value } } } } </code></pre></div><p>After that, all we have to do is delete values passed from the top of the stack as function parameters, thereby moving the stack. We have copied their values into a variable array.</p> <div class="language- extra-class"><pre class="language-text"><code>if block.Type == ObjFunc { start -= len(block.Info.(*FuncInfo).Params) } </code></pre></div><p>When a bytecode command loop finished, we must clear the stack correctly.</p> <div class="language- extra-class"><pre class="language-text"><code>last := rt.blocks[len(rt.blocks)-1] </code></pre></div><p>Delete the current block from the stack of blocks.</p> <div class="language- extra-class"><pre class="language-text"><code>rt.blocks = rt.blocks[:len(rt.blocks)-1] if status == statusReturn { </code></pre></div><p>If successfully exited from a function already executed, we will add the return value to the end of the previous stack.</p> <div class="language- extra-class"><pre class="language-text"><code> if last.Block.Type == ObjFunc { for count := len(last.Block.Info.(*FuncInfo).Results); count > 0; count-- { rt.stack[start] = rt.stack[len(rt.stack)-count] start++ } status = statusNormal } else { </code></pre></div><p>As you can see, if we do not execute the function, then we will not restore the stack status and exit the function as is. The reason is that loops and conditional structures that have been executed in the function are also bytecode blocks.</p> <div class="language- extra-class"><pre class="language-text"><code> return } } rt.stack = rt.stack[:start] </code></pre></div><h3 id="other-functions-for-operations-with-vm"><a href="#other-functions-for-operations-with-vm" class="header-anchor">#</a> Other functions for operations with VM </h3> <p>Your may create a virtual machine with the <strong>NewVM</strong> function. Each virtual machine will be added with four functions, such as <strong>ExecContract</strong>, <strong>MemoryUsage</strong>, <strong>CallContract</strong>, and <strong>Settings</strong>, through the <strong>Extend</strong> function.</p> <div class="language- extra-class"><pre class="language-text"><code>for key, item := range ext.Objects { fobj := reflect.ValueOf(item).Type() </code></pre></div><p>We traverse all the objects passed and only look at the functions.</p> <div class="language- extra-class"><pre class="language-text"><code> switch fobj.Kind() { case reflect.Func: </code></pre></div><p>We fill the <strong>ExtFuncInfo</strong> structure according to the information received about the function, and add its structure to the top level map <strong>Objects</strong> by name.</p> <div class="language- extra-class"><pre class="language-text"><code> data := ExtFuncInfo{key, make([]reflect.Type, fobj.NumIn()), make([]reflect.Type, fobj.NumOut()), make([]string, fobj.NumIn()), fobj.IsVariadic(), item} for i := 0; i <fobj.NumIn(); i++ { </code></pre></div><p>The <strong>ExtFuncInfo</strong> structure has an <strong>Auto</strong> parameter array. Usually the first parameter is <code>sc *SmartContract</code> or <code>rt *Runtime</code>, we cannot pass them from theNeedle language, because they are necessary for us to execute some golang functions. Therefore, we specify that these variables will be used automatically when these functions are called. In this case, the first parameter of the above four functions is <code>rt *Runtime</code>.</p> <div class="language- extra-class"><pre class="language-text"><code> if isauto, ok := ext.AutoPars[fobj.In(i).String()]; ok { data.Auto[i] = isauto } </code></pre></div><p>Information about assigning the parameters.</p> <div class="language- extra-class"><pre class="language-text"><code> data.Params[i] = fobj.In(i) } </code></pre></div><p>And the types of return values.</p> <div class="language- extra-class"><pre class="language-text"><code>for i := 0; i <fobj.NumOut(); i++ { data.Results[i] = fobj.Out(i) } </code></pre></div><p>Adds a function to the root <strong>Objects</strong> so that the compiler can find them later when using the contract.</p> <div class="language- extra-class"><pre class="language-text"><code> vm.Objects[key] = &ObjInfo{ObjExtFunc, data} } } </code></pre></div><h2 id="compiler"><a href="#compiler" class="header-anchor">#</a> Compiler </h2> <p>Functions in the compile.go file are responsible for compiling the array of tokens obtained from the lexical analyzer. Compilation can be divided into two levels conditionally. At the top level, we deal with functions, contracts, code blocks, conditional and loop statements, variable definitions, and so on. At the lower level, we compile expressions in code blocks or conditions in loops and conditional statements.</p> <p>First, we will start from the simple lower level. In the <strong>compileEval</strong> function, expressions can be converted to bytecode. Since we use a virtual machine with a stack, it is necessary to convert ordinary infix record expressions into postfix notation or reverse Polish notation. For example, we convert <code>1+2</code> to <code>12+</code> and put <code>1</code> and <code>2</code> to the stack. Then, we apply the addition operation to the last two elements in the stack and write the result to the stack. You can find this <a href="https://master.virmandy.net/perevod-iz-infiksnoy-notatsii-v-postfiksnuyu-obratnaya-polskaya-zapis/" target="_blank" rel="noopener noreferrer">conversion<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> algorithm on the Internet.</p> <p>The global variable <code>opers = map [uint32] operPrior</code> contains the priority of operations required for conversion to inverse Polish notation.</p> <p>The following variables are defined at the beginning of the <strong>compileEval</strong> function:</p> <ul><li><strong>buffer</strong> - temporary buffer for bytecode commands;</li> <li><strong>bytecode</strong> - final buffer of bytecode commands;</li> <li><strong>parcount</strong> - temporary buffer used to calculate parameters when calling a function;</li> <li><strong>setIndex</strong> - variables in the work process will be set to true when we assign map or array elements. For example, <code>a["my"] = 10</code>. In this case, we need to use the specified <strong>cmdSetIndex</strong> command.</li></ul> <p>We get a token in a loop and process it accordingly. For example, expression paring will be stopped if braces are found. When moving the string, we check whether the previous statement is an operation and whether it is inside the parentheses, otherwise it will exit the expression is parsed.</p> <div class="language- extra-class"><pre class="language-text"><code>case isRCurly, isLCurly: i-- if prevLex == isComma || prevLex == lexOper { return errEndExp } break main case lexNewLine: if i > 0 && ((*lexems)[i-1].Type == isComma || (*lexems)[i-1].Type == lexOper) { continue main } for k := len(buffer) - 1; k >= 0; k-- { if buffer[k].Cmd == cmdSys { continue main } } break main </code></pre></div><p>In general, the algorithm itself corresponds to an algorithm for converting to inverse Polish notation. With the consideration of the calling of necessary contracts, functions, and indexes, as well as other things not encountered during parsing and options for parsing lexIdent type tokens, then, variables, functions or contracts with this name will be checked. If nothing is found and this is not a function or contract call, then it will indicate an error.</p> <div class="language- extra-class"><pre class="language-text"><code>objInfo, tobj := vm.findObj(lexem.Value.(string), block) if objInfo == nil && (!vm.Extern || i> *ind || i >= len(*lexems)-2 || (*lexems)[i+1].Type != isLPar) { return fmt.Errorf(`unknown identifier %s`, lexem.Value.(string)) } </code></pre></div><p>We may encounter such a situation, and the contract call will be described later. In this example, if no functions or variables with the same name are found, then we think it is necessary to call a contract. In this compiled language, there is no difference between contracts and function calls. But we need to call the contract through the <strong>ExecContract</strong> function used in the bytecode.</p> <div class="language- extra-class"><pre class="language-text"><code>if objInfo.Type == ObjContract { if objInfo.Value != nil { objContract = objInfo.Value.(*Block) } objInfo, tobj = vm.findObj(`ExecContract`, block) isContract = true } </code></pre></div><p>We record the number of variables so far in <code>count</code>, which will also be written to the stack along with the number of function parameters. In each subsequent detection of parameters, we only need to increase this number by one unit at the last element of the stack.</p> <div class="language- extra-class"><pre class="language-text"><code>count := 0 if (*lexems)[i+2].Type != isRPar { count++ } </code></pre></div><p>We have a list Used of called parameters for contracts, then we need to mark the case of the contract is called. If the contract is called without parameters, we must add two empty parameters to call <strong>ExecContract</strong> to get at least two parameters.</p> <div class="language- extra-class"><pre class="language-text"><code>if isContract { name := StateName((*block)[0].Info.(uint32), lexem.Value.(string)) for j := len(*block) - 1; j >= 0; j-- { topblock := (*block)[j] if topblock.Type == ObjContract { if topblock.Info.(*ContractInfo).Used == nil { topblock.Info.(*ContractInfo).Used = make(map[string]bool) } topblock.Info.(*ContractInfo).Used[name] = true } } bytecode = append(bytecode, &ByteCode{cmdPush, name}) if count == 0 { count = 2 bytecode = append(bytecode, &ByteCode{cmdPush, ""}) bytecode = append(bytecode, &ByteCode{cmdPush, ""}) } count++ } </code></pre></div><p>If we see that there is a square bracket next, then we add the <strong>cmdIndex</strong> command to get the value by the index.</p> <div class="language- extra-class"><pre class="language-text"><code>if (*lexems)[i+1].Type == isLBrack { if objInfo == nil || objInfo.Type != ObjVar { return fmt.Errorf(`unknown variable %s`, lexem.Value.(string)) } buffer = append(buffer, &ByteCode{cmdIndex, 0}) } </code></pre></div><p>The <strong>CompileBlock</strong> function can generate object trees and expression-independent bytecodes. The compilation process is based on a finite state machine, just like a lexical analyzer, but with the following differences. First, we do not use symbols but tokens; second, we will immediately describe the <em>states</em> variables in all states and transitions. It represents an array of objects indexed by token type. Each token has a structure of <em>compileState</em>, and a new state is specified in <em>NewState</em>. If it is clear what structure we have resolved, we can specify the function of the handler in the <em>Func</em> field.</p> <p>Let us review the main state as an example.</p> <p>If we encounter a newline or comment, then we will remain in the same state. If we encounter the <strong>contract</strong> keyword, then we change the state to <em>stateContract</em> and start parsing the structure. If we encounter the <strong>func</strong> keyword, then we change the state to <em>stateFunc</em>. If other tokens are received, the function generating error will be called.</p> <div class="language- extra-class"><pre class="language-text"><code>{// stateRoot lexNewLine: {stateRoot, 0}, lexKeyword | (keyContract << 8): {stateContract | statePush, 0}, lexKeyword | (keyFunc << 8): {stateFunc | statePush, 0}, lexComment: {stateRoot, 0}, 0: {errUnknownCmd, cfError}, }, </code></pre></div><p>Suppose we encountered the <strong>func</strong> keyword and we have changed the state to <em>stateFunc</em>. Since the function name must follow the <strong>func</strong> keyword, we will keep the same state when changing the function name. For all other tokens, we will generate corresponding errors. If we get the function name in the token identifier, then we go to the <em>stateFParams</em> state, where we can get the parameters of the function.</p> <div class="language- extra-class"><pre class="language-text"><code>{// stateFunc lexNewLine: {stateFunc, 0}, lexIdent: {stateFParams, cfNameBlock}, 0: {errMustName, cfError}, }, </code></pre></div><p>At the same time as the above operations, we will call the <strong>fNameBlock</strong> function. It should be noted that the Block structure is created with the statePush mark, where we get it from the buffer and fill it with the data we need. The <strong>fNameBlock</strong> function is suitable for contracts and functions (including those nested in them). It fills the <em>Info</em> field with the corresponding structure and writes itself into the <em>Objects</em> of the parent block. In this way, we can call the function or contract with the specified name. Similarly, we create corresponding functions for all states and variables. These functions are usually very small and perform some duties when constructing the virtual machine tree.</p> <div class="language- extra-class"><pre class="language-text"><code>func fNameBlock(buf *[]*Block, state int, lexem *Lexem) error { var itype int prev := (*buf)[len(*buf)-2] fblock := (*buf)[len(*buf)-1] name := lexem.Value.(string) switch state { case stateBlock: itype = ObjContract name = StateName((*buf)[0].Info.(uint32), name) fblock.Info = &ContractInfo{ID: uint32(len(prev.Children) - 1), Name: name, Owner: (*buf)[0].Owner} default: itype = ObjFunc fblock.Info = &FuncInfo{} } fblock.Type = itype prev.Objects[name] = &ObjInfo{Type: itype, Value: fblock} return nil } </code></pre></div><p>For the <strong>CompileBlock</strong> function, it just traverses all the tokens and switches states according to the tokens described in states. Almost all additional tokens correspond to additional program codes.</p> <ul><li><strong>statePush</strong> – adds the <strong>Block</strong> object to the object tree;</li> <li><strong>statePop</strong> - used when the block ends with a closing brace;</li> <li><strong>stateStay</strong> - you need to keep the current mark when changing to a new state;</li> <li><strong>stateToBlock</strong> - transition to the <strong>stateBlock</strong> state for processing <em>while</em> and <em>if</em>. After processing expressions, you need to process blocks within the braces;</li> <li><strong>stateToBody</strong> - transition to the <strong>stateBody</strong> state;</li> <li><strong>stateFork</strong> - save the marked position. When the expression starts with an identifier or a name with <code>$</code>, we can make function calls or assignments;</li> <li><strong>stateToFork</strong> – used to get the token stored in <strong>stateFork</strong>, which will be passed to the process function;</li> <li><strong>stateLabel</strong> – used to insert <strong>cmdLabel</strong> commands. <em>while</em> structure requires this flag;</li> <li><strong>stateMustEval</strong> – check the availability of conditional expressions at the beginning of <em>if</em> and <em>while</em> structures.</li></ul> <p>In addition to the <strong>CompileBlock</strong> function, the <strong>FlushBlock</strong> function should also be mentioned. But the problem is that the block tree is constructed independently of existing virtual machines. More precisely, we obtain information about functions and contracts that exist in a virtual machine, but we collect the compiled blocks into a separate tree. Otherwise, if an error occurs during compilation, we must roll back the virtual machine to the previous state. Therefore, we go to the compilation tree separately, but after the compilation is successful, the <strong>FlushContract</strong> function must be called. This function adds the completed block tree to the current virtual machine. The compilation phase is now complete.</p> <h2 id="lexical-analyzer"><a href="#lexical-analyzer" class="header-anchor">#</a> Lexical analyzer </h2> <p>The lexical analyzer processes incoming strings and forms a sequence of tokens of the following types :</p> <ul><li><strong>lexSys</strong> - system token, for example: <code>{}, [], (), ,, .</code> etc;</li> <li><strong>lexOper</strong> - operation token, for example: <code>+, -, /, \, *</code>;</li> <li><strong>lexNumber</strong> - number;</li> <li><strong>lexident</strong> - identifier;</li> <li><strong>lexNewline</strong> - newline character;</li> <li><strong>lexString</strong> - string;</li> <li><strong>lexComment</strong> - comment;</li> <li><strong>lexKeyword</strong> - keyword;</li> <li><strong>lexType</strong> - type;</li> <li><strong>lexExtend</strong> - reference to external variables or functions, for example: <code>$myname</code>.</li></ul> <p>In the current version, a conversion table (finite state machine) is initially constructed with the help of the <a href="#lextable-lextable-go">lextable.go</a> file to parse the tokens, which is written to the lex_table.go file. In general, you can get rid of the conversion table initially generated by the file and create a conversion table in the memory (<code>init()</code>) immediately upon startup. The lexical analysis itself occurs in the lexParser function in the <a href="#lex-go">lex.go</a> file.</p> <h3 id="lextable-lextable-go"><a href="#lextable-lextable-go" class="header-anchor">#</a> lextable/lextable.go </h3> <p>Here we define the alphabet to operate and describe how the finite state machine changes from one state to another based on the next received symbol.</p> <p><em>states</em> is a JSON object containing a list of states.</p> <p>Except for specific symbols, <code>d</code> stands for all symbols not specified in the state. <code>n</code> stands for 0x0a, <code>s</code> stands for space, <code>q</code> stands for backquote, <code>Q</code> stands for double quote, <code>r</code> stands for character >= 128, <code>a</code> stands for AZ and az, and <code>1</code> stands for 1- 9.</p> <p>The name of these states are keys, and the possible values are listed in the value object. Then, there is a new state to make transitions for each group. Then there is the name of the token. If we need to return to the initial state, the third parameter is the service token, which indicates how to handle the current symbol.</p> <p>For example, we have the main state and the incoming characters <code>/</code>, <code>"/": ["solidus", "", "push next"]</code>,</p> <ul><li><p><strong>push</strong> - gives the command to remember that it is in a separate stack ;</p></li> <li><p><strong>next</strong> - goes to the next character, and at the same time we change the status to <strong>solidus</strong>. After that, gets the next character and check the status of <strong>solidus</strong>.</p></li></ul> <p>If the next character has <code>/</code> or <code>/*</code>, then we go to the comment <strong>comment</strong> state because they start with <code>//</code> or <code>/*</code>. Obviously, each comment has a different state afterwards, because they end with a different symbol.</p> <p>If the next character is not <code>/</code> and <code>*</code>, then we record everything in the stack as <strong>lexOper</strong> type tags, clear the stack and return to the main state.</p> <p>The following module converts the state tree into a numeric array and writes it into the <em>lex_table.go</em> file.</p> <p>In the first loop:</p> <p>We form an alphabet of valid symbols.</p> <div class="language- extra-class"><pre class="language-text"><code>for ind, ch := range alphabet { i := byte(ind) </code></pre></div><p>In addition, in <strong>state2int</strong>, we provide each state with its own sequence identifier.</p> <div class="language- extra-class"><pre class="language-text"><code> state2int := map[string]uint{`main`: 0} if err := json.Unmarshal([]byte(states), &data); err == nil { for key := range data { if key != `main` { state2int[key] = uint(len(state2int)) </code></pre></div><p>When we traverse all states and each set in a state and each symbol in a set, we write a three-byte number [new state identifier (0 = main)] + [token type ( 0-no token)] + [token]. The bidimensionality of the <em>table</em> array is that it is divided into states and 34 input symbols from the <em>alphabet</em> array, which are arranged in the same order.</p> <p>We are in the <em>main</em> state on the zero row of the <em>table</em>. Take the first character, find its index in the <em>alphabet</em> array, and get the value from the column with the given index. Starting from the value received, we receive the token in the low byte. If the parsing is complete, the second byte indicates the type of token received. In the third byte, we receive the index of the next new state. All of these are described in more detail in the <strong>lexParser</strong> function in <em>lex.go</em>.</p> <p>If you want to add some new characters, you need to add them to the <em>alphabet</em> array and increase the quantity of the <em>AlphaSize</em> constant. If you want to add a new symbol combination, it should be described in the status, similar to the existing options. After the above operation, run the <em>lextable.go</em> file to update the <em>lex_table.go</em> file.</p> <h3 id="lex-go"><a href="#lex-go" class="header-anchor">#</a> lex-go </h3> <p>The <strong>lexParser</strong> function directly generates lexical analysis and returns an array of received tags based on incoming strings. Let us analyze the structure of tokens.</p> <div class="language- extra-class"><pre class="language-text"><code>type Lexem struct { Type uint32 // Type of the lexem Value interface{} // Value of lexem Line uint32 // Line of the lexem Column uint32 // Position inside the line } </code></pre></div><ul><li><p><strong>Type</strong> - token type. It has one of the following values: <code>lexSys, lexOper, lexNumber, lexIdent, lexString, lexComment, lexKeyword, lexType, lexExtend</code>;</p></li> <li><p><strong>Value</strong> – token value. The type of value depends on the token type, Let us analyze it in more detail:</p> <ul><li><strong>lexSys</strong> - includes brackets, commas, etc. In this case, <code>Type = ch << 8 | lexSys</code>, please refer to the <code>isLPar ... isRBrack</code> constant, and its value is uint32 bits;</li> <li><strong>lexOper</strong> - the value represents an equivalent character sequence in the form of uint32. See the <code>isNot ... isOr</code> constants;</li> <li><strong>lexNumber</strong> - numbers are stored as int64 or float64. If the number has a decimal point, it is float64;</li> <li><strong>lexIdent</strong> - identifiers are stored as string;</li> <li><strong>lexNewLine</strong> - newline character. Also used to calculate the row and token position;</li> <li><strong>lexString</strong> - lines are stored as string;</li> <li><strong>lexComment</strong> - comments are stored as string;</li> <li><strong>lexKeyword</strong> - for keywords, only the corresponding indexes are stored, see the <code>keyContract ... keyTail</code> constant. In this case <code>Type = KeyID << 8 | lexKeyword</code>. In addition, it should be noted that the <code>true, false, nil</code> keywords will be immediately converted to lexNumber type tokens, and the corresponding <code>bool</code> and <code>intreface {}</code> types will be used;</li> <li><strong>lexType</strong> – this value contains the corresponding <code>reflect.Type</code> type value;</li> <li><strong>lexExtend</strong> – identifiers beginning with a <code>$</code>. These variables and functions are passed from the outside and are therefore assigned to special types of tokens. This value contains the name as a string without a $ at the beginning.</li></ul></li> <li><p><strong>Line</strong> - the line where the token is found;</p></li> <li><p><strong>Column</strong> - in-line position of the token.</p></li></ul> <p>Let us analyze the <strong>lexParser</strong> function in detail. The <strong>todo</strong> function looks up the symbol index in the alphabet based on the current state and the incoming symbol, and obtains a new state, token identifier (if any), and other tokens from the conversion table. The parsing itself involves calling the <strong>todo</strong> function in turn for each next character and switching to a new state. Once the tag is received, we create the corresponding token in the output criteria and continue the parsing process. It should be noted that during the parsing process, we do not accumulate the token symbols into a separate stack or array, because we only save the offset of the start of the token. After getting the token, we move the offset of the next token to the current parsing position.</p> <p>All that remains is to check the lexical status tokens used in the parsing:</p> <ul><li><strong>lexfPush</strong> - this token means that we start to accumulate symbols in a new token;</li> <li><strong>lexfNext</strong> - the character must be added to the current token;</li> <li><strong>lexfPop</strong> - the receipt of the token is complete. Usually, with this flag we have the identifier type of the parsed token;</li> <li><strong>lexfSkip</strong> - this token is used to exclude characters from parsing. For example, the control slashes in the string are \n \r ". They will be automatically replaced during the lexical analysis stage.</li></ul> <h2 id="needle-language"><a href="#needle-language" class="header-anchor">#</a> Needle language </h2> <h3 id="lexemes"><a href="#lexemes" class="header-anchor">#</a> Lexemes </h3> <p>The source code of a program must be in UTF-8 encoding.</p> <p>The following lexical types are processed:</p> <ul><li><strong>Keywords</strong> - <code>action, break, conditions, continue, contract, data, else, error, false, func, If, info, nil, return, settings, true, var, warning, while</code>;</li> <li><strong>Number</strong> - only decimal numbers are accepted. There are two basic types: <strong>int</strong> and <strong>float</strong>. If the number has a decimal point, it becomes a float <strong>float</strong>. <strong>int</strong> type is equivalent to <strong>int64</strong> in golang, while <strong>float</strong> type is equivalent to <strong>float64</strong> in golang.</li> <li><strong>String</strong> - the string can be enclosed in double quotes <code>("a string")</code> or backquotes <code>(\`a string\`)</code>. Both types of strings can contain newline characters. Strings in double quotes can contain double quotes, newline characters, and carriage returns escaped with slashes. For example, <code>"This is a \"first string\".\r\nThis is a second string."</code>.</li> <li><strong>Comment</strong> - there are two types of comments. Single-line comments use two slashes (//). For example, // This is a single-line comment. Multi-line comments use slash and asterisk symbols and can span multiple lines. For example, <code>/* This is a multi-line comment */</code>.</li> <li><strong>Identifier</strong> - the names of variables and functions composed of a-z and A-Z letters, UTF-8 symbols, numbers and underscores. The name can start with a letter, underscore, <code>@</code> or <code>$</code>. The name starting with <code>$</code> is the name of the variable defined in the <strong>data section</strong>. The name starting with <code>$</code> can also be used to define global variables in the scope of <strong>conditions</strong> and <strong>action sections</strong>. Ecosystem contracts can be called using the <code>@</code> symbol. For example: <code>@1NewTable(...)</code>.</li></ul> <h3 id="types"><a href="#types" class="header-anchor">#</a> Types </h3> <p>Corresponding golang types are specified next to theNeedle types.</p> <ul><li><strong>bool</strong> - bool, <strong>false</strong> by default;</li> <li><strong>bytes</strong> - []byte{}, an empty byte array by default;</li> <li><strong>int</strong> - int64, <strong>0</strong> by default;</li> <li><strong>address</strong> - uint64, <strong>0</strong> by default;</li> <li><strong>array</strong> - []interface{}, an empty array by default;</li> <li><strong>map</strong> - map[string]interface{}, an empty object array by default;</li> <li><strong>money</strong> - decimal. Decimal, <strong>0</strong> by default;</li> <li><strong>float</strong> - float64, <strong>0</strong> by default;</li> <li><strong>string</strong> - string, an empty string by default;</li> <li><strong>file</strong> - map[string]interface{}, an empty object array by default.</li></ul> <p>These types of variables are defined with the <code>var</code> keyword. For example, <code>var var1, var2 int</code>. When defined in this way, a variable will be assigned with a default value by type.</p> <p>All variable values are of the interface{} type, and then they are assigned to the required golang types. Therefore, for example, array and map types are golang types []interface{} and map[string]interface{}. Both types of arrays can contain elements of any type.</p> <h3 id="expressions"><a href="#expressions" class="header-anchor">#</a> Expressions </h3> <p>An expression may include arithmetic operations, logical operations, and function calls. All expressions are evaluated from left to right by priority of operators. If having an equal priority, operators are evaluated from left to right.</p> <p>Priority of operations from high to low:</p> <ul><li><strong>Function call and parentheses</strong> - when a function is called, passed parameters will be calculated from left to right;</li> <li><strong>Unary Operation</strong> - logical negation <code>!</code> and arithmetic sign change <code>-</code>;</li> <li><strong>Multiplication and Division</strong> - arithmetic multiplication <code>*</code> and division <code>/</code>;</li> <li><strong>Addition and Subtraction</strong> - arithmetic addition <code>+</code> and subtraction <code>-</code>;</li> <li><strong>Logical comparison</strong> - <code>>=>> >=</code>;</li> <li><strong>Logical equality and inequality</strong> - <code>== !=</code>;</li> <li><strong>Logical AND</strong> - <code>&&</code>;</li> <li><strong>Logical OR</strong> - <code>||</code>.</li></ul> <p>When evaluating logical AND and OR, both sides of the expression are evaluated in any case.</p> <p>Needle has no type checking during compilation. When evaluating operands, an attempt is made to convert the type to a more complex type. The type of complexity order can be as follows: <code>string, int, float, money</code>. Only part of the type conversions is implemented. The string type supports addition operations, and the result will be string concatenation. For example, <code>string + string = string, money-int = money, int * float = float</code>.</p> <p>For functions, type checking is performed on the <code>string</code> and <code>int</code> types during execution.</p> <p><strong>array</strong> and <strong>map</strong> types can be addressed by index. For the <strong>array</strong> type, the <strong>int</strong> value must be specified as the index. For the <strong>map</strong> type, a variable or <strong>string</strong> value must be specified. If you assign a value to an <strong>array</strong> element whose index is greater than the current maximum index, an empty element will be added to the array. The initial value of these elements is <strong>nil</strong>. For example: .. code:</p> <div class="language- extra-class"><pre class="language-text"><code>var my array my[5] = 0 var mymap map mymap["index"] = my[3] </code></pre></div><p>In expressions of conditional logical values (such as <code>if, while, &&, ||, !</code>), the type is automatically converted to a logical value. If the type is not the default value, it is true.</p> <div class="language- extra-class"><pre class="language-text"><code>var mymap map var val string if mymap && val { ... } </code></pre></div><h3 id="scope"><a href="#scope" class="header-anchor">#</a> Scope </h3> <p>Braces specify a block that can contain local scope variables. By default, the scope of a variable extends to its own blocks and all nested blocks. In a block, you can define a new variable using the name of an existing variable. However, in this case, external variables with the same name become unavailable.</p> <div class="language- extra-class"><pre class="language-text"><code> var a int a = 3 { var a int a = 4 Println(a) // 4 } Println(a) // 3 </code></pre></div><h3 id="contract-execution"><a href="#contract-execution" class="header-anchor">#</a> Contract execution </h3> <p>When calling a contract, parameters defined in <strong>data</strong> must be passed to it. Before executing a contract, the virtual machine receives these parameters and assigns them to the corresponding variables ($Param). Then, the predefined <strong>conditions</strong> function and <strong>action</strong> function are called.</p> <p>Errors that occur during contract execution can be divided into two types: form errors and environment errors. Form errors are generated using special commands: <code>error, warning, info</code> and when the built-in function returns <code>err</code> not equal to <em>nil</em>.</p> <p>The Needle language does not handle exceptions. Any error will terminate the execution of contracts. Since a separate stack and structure for saving variable values are created when a contract is executed, the golang garbage collection mechanism will automatically delete these data when a contract is executed.</p> <h3 id="backus-naur-form-bnf"><a href="#backus-naur-form-bnf" class="header-anchor">#</a> Backus–Naur Form (BNF) </h3> <p>In computer science, BNF is a notation technique for context-free syntax and is usually used to describe the syntax of the language used in computing.</p> <ul><li><decimal digit></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9' </code></pre></div><ul><li><decimal number></li></ul> <div class="language- extra-class"><pre class="language-text"><code><decimal digit> {<decimal digit>} </code></pre></div><ul><li><symbol code></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'''<any symbol>''' </code></pre></div><ul><li><real number></li></ul> <div class="language- extra-class"><pre class="language-text"><code>['-'] <decimal number'.'[<decimal number>] </code></pre></div><ul><li><integer number></li></ul> <div class="language- extra-class"><pre class="language-text"><code>['-'] <decimal number> | <symbol code> </code></pre></div><ul><li><number></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'<integer number> | <real number>' </code></pre></div><ul><li><letter></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'A' |'B' | ... |'Z' |'a' |'b' | ... |'z' | 0x80 | 0x81 | ... | 0xFF </code></pre></div><ul><li><space></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'0x20' </code></pre></div><ul><li><tabulation></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'0x09' </code></pre></div><ul><li><newline></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'0x0D 0x0A' </code></pre></div><ul><li><special symbol></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'!' |'"' |'$' |''' |'(' |')' |'\*' |'+' |',' |'-' |'.' |'/ '|'<' |'=' |'>' |'[' |'\\' |']' |'_' |'|' |'}' | '{' | <tabulation> | <space> | <newline> </code></pre></div><ul><li><symbol></li></ul> <div class="language- extra-class"><pre class="language-text"><code><decimal digit> | <letter> | <special symbol> </code></pre></div><ul><li><name></li></ul> <div class="language- extra-class"><pre class="language-text"><code>(<letter> |'_') {<letter> |'_' | <decimal digit>} </code></pre></div><ul><li><function name></li></ul> <div class="language- extra-class"><pre class="language-text"><code><name> </code></pre></div><ul><li><variable name></li></ul> <div class="language- extra-class"><pre class="language-text"><code><name> </code></pre></div><ul><li><type name></li></ul> <div class="language- extra-class"><pre class="language-text"><code><name> </code></pre></div><ul><li><string symbol></li></ul> <div class="language- extra-class"><pre class="language-text"><code><tabulation> | <space> |'!' |'#' | ... |'[' |']' | ... </code></pre></div><ul><li><string element></li></ul> <div class="language- extra-class"><pre class="language-text"><code>{<string symbol> |'\"' |'\n' |'\r'} </code></pre></div><ul><li><string></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'"' {<string element>}'"' |'\`' {<string element>}'\`' </code></pre></div><ul><li><assignment operator></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'=' </code></pre></div><ul><li><unary operator></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'-' </code></pre></div><ul><li><binary operator></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'==' |'!=' |'>' |'<' |'<=' |'>=' |'&&' |'||' |'\*' |'/' |'+ '|'-' </code></pre></div><ul><li><operator></li></ul> <div class="language- extra-class"><pre class="language-text"><code><assignment operator> | <unary operator> | <binary operator> </code></pre></div><ul><li><parameters></li></ul> <div class="language- extra-class"><pre class="language-text"><code><expression> {','<expression>} </code></pre></div><ul><li><contract call></li></ul> <div class="language- extra-class"><pre class="language-text"><code><contract name>'(' [<parameters>]')' </code></pre></div><ul><li><function call></li></ul> <div class="language- extra-class"><pre class="language-text"><code><contract call> [{'.' <name>'(' [<parameters>]')'}] </code></pre></div><ul><li><block contents></li></ul> <div class="language- extra-class"><pre class="language-text"><code><block command> {<newline><block command>} </code></pre></div><ul><li><block></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'{'<block contents>'}' </code></pre></div><ul><li><block command></li></ul> <div class="language- extra-class"><pre class="language-text"><code>(<block> | <expression> | <variables definition> | <if> | <while> | break | continue | return) </code></pre></div><ul><li><if></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'if <expression><block> [else <block>]' </code></pre></div><ul><li><while></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'while <expression><block>' </code></pre></div><ul><li><contract></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'contract <name> '{'[<data section>] {<function>} [<conditions>] [<action>]'}'' </code></pre></div><ul><li><data section></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'data '{' {<data parameter><newline>} '}'' </code></pre></div><ul><li><data parameter></li></ul> <div class="language- extra-class"><pre class="language-text"><code><variable name> <type name>'"'{<tag>}'"' </code></pre></div><ul><li><tag></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'optional | image | file | hidden | text | polymap | map | address | signature:<name>' </code></pre></div><ul><li><conditions></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'conditions <block>' </code></pre></div><ul><li><action></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'action <block>' </code></pre></div><ul><li><function></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'func <function name>'('[<variable description>{','<variable description>}]')'[{<tail>}] [<type name>] <block>' </code></pre></div><ul><li><variable description></li></ul> <div class="language- extra-class"><pre class="language-text"><code><variable name> {',' <variable name>} <type name> </code></pre></div><ul><li><tail></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'.'<function name>'('[<variable description>{','<variable description>}]')' </code></pre></div><ul><li><variables definition></li></ul> <div class="language- extra-class"><pre class="language-text"><code>'var <variable description>{','<variable description>}' </code></pre></div></div> <footer class="page-edit"><div class="edit-link"><a href="https://github.com/IBAX-io/documentation/edit/master/docs/topics/vm.md" target="_blank" rel="noopener noreferrer">Edit this page on GitHub</a> <span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></div> <div class="last-updated"><span class="prefix">Last Updated:</span> <span class="time">7/6/2023, 6:59:03 PM</span></div></footer> <div class="page-nav"><p class="inner"><span class="prev"> ← <a href="/topics/templates2.html" class="prev"> Template Language </a></span> <span class="next"><a href="/topics/daemons.html"> Daemon </a> → </span></p></div> </main></div><div class="global-ui"></div></div> <script src="/assets/js/app.d049f7ab.js" defer></script><script src="/assets/js/2.8d94a0db.js" defer></script><script src="/assets/js/130.59e12a7f.js" defer></script> </body> </html>