netSerial

serializer producing compact bitstreams
git clone https://noulin.net/git/netSerial.git
Log | Files | Refs | README | LICENSE

README.md (2840B)


      1 # Sheepy
      2 This is a sheepy package for [sheepy](https://spartatek.se/r/sheepy/file/README.md.html) and using [libsheepy](https://spartatek.se/r/libsheepy/file/README.md.html)
      3 
      4 # netSerial
      5 
      6 netSerial is a serializer/deserializer for JSON data structures that has compact binary format.
      7 
      8 In general, it gives smaller bitstreams than JSON strings and smallJson serializer bitstream.
      9 
     10 The netSerial serializer is in average 1.5 times slower than the smallJson serializer.
     11 
     12 # Usage
     13 
     14 Install with spm: `spm install netSerial`
     15 
     16 Include netSerial: `#include "shpPackages/netSerial/netSerial.h"`, then:
     17 
     18 ```c
     19 // create a netSerial object:
     20 createNetSerial(n);
     21 
     22 // create a JSON
     23 parseG(&n, "[3,270,-86942]");
     24 
     25 smallBytest *bitstream = serialG(&n);
     26 ```
     27 
     28 # Encoding
     29 
     30 Data types are encoded on 4 bits:
     31 - undefined
     32 - bool
     33 - dict
     34 - double
     35 - varint
     36 - string
     37 - array
     38 - bytes = faststring
     39 - packed dict
     40 - packed double
     41 - packed varint
     42 - packed string
     43 - packed array
     44 - packed bytes
     45 - dict  - all elements have same type
     46 - array - all elements have same type
     47 
     48 Integers are encoded as varint (n <<1 ^ n >> 63, see [protobuf](https://developers.google.com/protocol-buffers/docs/encoding)) and lengths (array, dictionary) are encoded as varuint.
     49 
     50 The integer _150_ is encoded as `0xc4 0x25`.
     51 
     52 - 0xc_4_ is integer type
     53 - 0x_c_4 are the first 3 bits in 150, bit 7 is set so next byte represents the 7 next bits in 150
     54 - _0x25_ are last bits in 150
     55 
     56 The encoding of `[3,270,-86942]` gives:
     57 ```
     58 0x4f           - uniform array of varint elements
     59 0x03           - 3 elements
     60 0x06           - int 3
     61 0x9c 0x04      - int 270
     62 0xbb 0xce 0x0a - int -86942
     63 ```
     64 
     65 The arrays have 2 variants: normal and uniform.
     66 
     67 Normal arrays have elements of any types whereas uniform arrays have only one type of elements.
     68 
     69 For example, `[null,null,null]` is a uniform array of undefined values, encoded as:
     70 ```
     71 0x0f - uniform array of undefined(null) elements
     72 0x03 - 3 elements
     73 ```
     74 
     75 undefined is a type without data, only the element count is needed to represent an array of null elements.
     76 
     77 The bool type is encoded in 4bits type + 1 bit data and is packed by default:
     78 
     79 ```
     80 {"sdf":true,"0":null,"1":null,"2":true,"3":true}
     81 
     82 0x52                - dictionary with 5 elements
     83 0x73 0x64 0x66 0x00 - "sdf" string
     84 0x71                - bool type and bool data for bool 1,2 and 3 (true)
     85 0x30 0x00           - "0" string
     86 0x00                - undefined 1 and 2
     87 0x31 0x00           - "1" string
     88 0x32 0x00           - "2" string
     89 0x11                - bool type 2 and 3
     90 0x33 0x00           - "3" string
     91 ```
     92 
     93 Encoding a dictionary and an array:
     94 ```
     95 {"a":[3,270,-86942]}
     96 
     97 0xfe           - uniform dictionary of uniform arrays
     98 0x01           - 1 element
     99 0x61 0x00      - "a" string
    100 0x34           - int type and 3 elements
    101 0x06           - int 3
    102 0x9c 0x04      - int 270
    103 0xbb 0xce 0x0a - int -86942
    104 ```
    105 
    106 -