espressif/openthread
1
0
Fork 0
mirror of https://github.com/espressif/openthread.git synced 2026-10-20 22:48:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

spinel: Clarify data packing notation and fix implementation. (#1433)

Browse Source 
The data packing format that I designed for Spinel was a bit too
clever for its own good. While elegant from a certain perspective, the
behavior of the data blob (`D`), array (`A(...)`), and structure
(`T(...)`) types was way too confusing. I thought that I was (as the
guy who came up with it) immune from such confusion, but after taking
the time to go through all the code and address all of the mishaps in
the code it seems that the subtleties befuddled even its creator on a
time or two.

The specific issue that made me realize I needed to prioritize making
this fix was #1393.

As far as packing types go, this commit makes the following changes:

*   **New type `DATATYPE_DATA_WLEN` (`d`)**: Just like `DATATYPE_DATA`
    except that it ALWAYS prepends the length, irrespective of the
    type's location in the type string. Think of WLEN as being "With
    LENgth". Unless you are adding a property that has a type which is
    just a single data blob, you should use this type instead of
    `DATATYPE_DATA` (`D`). `DATATYPE_DATA` is still useful, so it will
    be sticking around.
*   **Changed type `DATATYPE_STRUCT` (was `T`, now `t`)**: The new
    struct type, ALWAYS prepends the length of the contents of the
    struct, just like `DATATYPE_DATA_WLEN`. The old `T` style struct
    still works and behaves the same way, but its use is now
    deprecated.
*   **Changed type `DATATYPE_ARRAY` (`A`)**: This type has changed in
    documentation only, as no one is currently using it in the way
    that it was originally specified. The change is that this type
    NEVER prepends the length of the entire array, and using it
    alongside other types is now unspecified (it should only be used
    alone).

I also added some convenient macros for building up datatype signature
strings that include structs.

One of the goals of these changes was to make this commit a change in
specification only, not to change what was currently on-the-wire.
However, after updating the spec and then going through and updating
all of the code to use the new nomenclature, I was horrified to find
that, due to the confusing nature of the previous datatype signature
string format, many properties that are arrays were not implemented
correctly --- making them impossible to update or add fields to
without breaking backward compatibility. To add insult to injury, some
of these instances were written by me!

This change updates all of the array properties to reflect how they
should have been implemented to begin with.
This commit is contained in:
Robert Quattlebaum
2017-03-10 11:48:31 -08:00
committed by Jonathan Hui
parent 6c3d7ad2c5
commit b98e3c70eb
15 changed files with 364 additions and 214 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/spinel-cli/spinel/codec.py
+69 -21
View File
@@ -84,8 +84,9 @@ TIMEOUT_PROP = 2
#'E': DATATYPE_EUI64: EUI-64 Address. (Big-endian)
#'e': DATATYPE_EUI48: EUI-48 Address. (Big-endian)
#'D': DATATYPE_DATA: Arbitrary Data. (See section 7.3)
#'d': DATATYPE_DATA_WLEN: Arbitrary Data with Prepended Length. (See section 7.3)
#'U': DATATYPE_UTF8: Zero-terminated UTF8-encoded string.
#'T': DATATYPE_STRUCT: Structured datatype. Compound type. (See section 7.4)
#'t': DATATYPE_STRUCT: Structured datatype. Compound type. Length prepended. (See section 7.4)
#'A': DATATYPE_ARRAY: Array of datatypes. Compound type. (See section 7.5)
@@ -123,11 +124,14 @@ class SpinelCodec(object):
def parse_e(cls, payload): return payload[:6]
@classmethod
def parse_U(cls, payload): return payload[:-1] # strip null
def parse_U(cls, payload): return payload[:payload.index('\0')] # strip null
@classmethod
def parse_D(cls, payload): return payload
@classmethod
def parse_d(cls, payload): return payload[2:unpack("<H", payload[:2])[0]]
@classmethod
def parse_i(cls, payload):
""" Decode EXI integer format. """
@@ -145,6 +149,27 @@ class SpinelCodec(object):
return (value, value_len + 1)
@classmethod
def parse_i_len(cls, payload):
""" Decode length of EXI integer format. """
return cls.parse_i(payload)[1];
@classmethod
def index_of_ending_brace(cls, spinel_format, idx):
""" Determines the index of the matching closing brace. """
count = 1
orig_idx = idx
while count > 0 and idx < len(spinel_format)-1:
idx += 1
if spinel_format[idx] == ')':
count -= 1
elif spinel_format[idx] == '(':
count += 1
if count != 0:
raise ValueError('Unbalanced parenthesis in format string "' + spinel_format + '", idx=' + idx)
return idx;
@classmethod
def parse_field(cls, payload, spinel_format):
map_decode = {
@@ -160,6 +185,7 @@ class SpinelCodec(object):
'e': cls.parse_e,
'U': cls.parse_U,
'D': cls.parse_D,
'd': cls.parse_d,
'i': cls.parse_i,
}
try:
@@ -169,7 +195,7 @@ class SpinelCodec(object):
return None
@classmethod
def get_payload_size(cls, spinel_format):
def get_payload_size(cls, payload, spinel_format):
map_lengths = {
'b': 1,
'c': 1,
@@ -189,16 +215,33 @@ class SpinelCodec(object):
while idx < len(spinel_format):
format = spinel_format[idx]
if format == 'T':
if format == 't':
if spinel_format[idx+1] != '(':
raise ValueError('Invalid structure format')
# TODO: count parentheses. There is a problem with nested structs.
struct_end = idx + spinel_format[idx:].index(')')
struct_format = spinel_format[idx+2:struct_end]
result += cls.get_payload_size(struct_format)
struct_end = cls.index_of_ending_brace(spinel_format, idx + 1);
result += 2 + cls.parse_S(payload[result:]);
idx = struct_end + 1
elif format == 'd':
result += 2 + cls.parse_S(payload[result:]);
idx += 1
elif format == 'D' or format == 'A':
if idx != len(spinel_format) - 1:
raise ValueError('Invalid type syntax for "' + format + '", must go at end of format string')
result = len(payload);
idx += 1
elif format == 'U':
result += payload[result:].index(0) + 1;
idx += 1
elif format == 'i':
result += cls.parse_i_len(payload[result:])
idx += 1
else:
result += map_lengths[format]
idx += 1
@@ -216,35 +259,36 @@ class SpinelCodec(object):
if format == 'A':
if spinel_format[idx+1] != '(':
raise ValueError('Invalid structure format')
# TODO: count parentheses. There is a problem with arrays placed one by one.
array_end = idx + spinel_format[idx:].rindex(')')
array_end = cls.index_of_ending_brace(spinel_format, idx + 1);
array_format = spinel_format[idx+2:array_end]
array = []
while len(payload):
array.append(cls.parse_fields(payload, array_format))
payload = payload[cls.get_payload_size(array_format):]
payload = payload[cls.get_payload_size(payload, array_format):]
result.append(tuple(array))
idx = array_end + 1
elif format == 'T':
elif format == 't':
if spinel_format[idx+1] != '(':
raise ValueError('Invalid structure format')
# TODO: count parentheses. There is a problem with nested structs.
struct_end = idx + spinel_format[idx:].index(')')
struct_end = cls.index_of_ending_brace(spinel_format, idx + 1);
struct_format = spinel_format[idx+2:struct_end]
struct_len = cls.get_payload_size(struct_format)
struct_len = cls.parse_S(payload);
result.append(cls.parse_fields(payload, struct_format))
payload = payload[struct_len:]
result.append(cls.parse_fields(payload[2:struct_len+2], struct_format))
payload = payload[struct_len+2:]
idx = struct_end + 1
else:
result.append(cls.parse_field(payload, format))
payload = payload[cls.get_payload_size(format):]
payload = payload[cls.get_payload_size(payload, format):]
idx += 1
@@ -298,6 +342,9 @@ class SpinelCodec(object):
@classmethod
def encode_D(cls, value): return value
@classmethod
def encode_d(cls, value): return cls.encode_S(len(value)) + value
@classmethod
def encode_field(cls, code, value):
map_encode = {
@@ -313,6 +360,7 @@ class SpinelCodec(object):
'e': cls.encode_e,
'U': cls.encode_U,
'D': cls.encode_D,
'd': cls.encode_d,
'i': cls.encode_i,
}
try:
@@ -434,7 +482,7 @@ class SpinelPropertyHandler(SpinelCodec):
def THREAD_PARENT(self, _wpan_api, payload): return self.parse_fields(payload, "ES")
def THREAD_CHILD_TABLE(self, _, payload): return self.parse_fields(payload, "A(T(ESLLCCcC))")
def THREAD_CHILD_TABLE(self, _, payload): return self.parse_fields(payload, "A(t(ESLLCCcC))")
def THREAD_LEADER_RID(self, _, payload): return self.parse_C(payload)
@@ -587,7 +635,7 @@ class SpinelPropertyHandler(SpinelCodec):
return self.parse_C(payload)
def THREAD_NEIGHBOR_TABLE(self, _, payload):
return self.parse_fields(payload, 'A(T(ESLCcCbLL))')
return self.parse_fields(payload, 'A(t(ESLCcCbLL))')
def THREAD_CONTEXT_REUSE_DELAY(self, _, payload):
return self.parse_L(payload)
@@ -640,7 +688,7 @@ class SpinelPropertyHandler(SpinelCodec):
def PIB_MAC_SECURITY_ENABLED(self, _wpan_api, payload): pass
def MSG_BUFFER_COUNTERS(self, _wpan_api, payload): return self.parse_fields(payload, "T(SSSSSSSSSSSSSSSS)")
def MSG_BUFFER_COUNTERS(self, _wpan_api, payload): return self.parse_fields(payload, "SSSSSSSSSSSSSSSS")
#=========================================
tools/spinel-cli/spinel/const.py
+9 -11
View File
@@ -103,7 +103,7 @@ class SPINEL(object):
PROP_MAC_SCAN_STATE = PROP_MAC__BEGIN + 0 # < [C]
PROP_MAC_SCAN_MASK = PROP_MAC__BEGIN + 1 # < [A(C)]
PROP_MAC_SCAN_PERIOD = PROP_MAC__BEGIN + 2 # < ms-per-channel [S]
# < chan,rssi,(laddr,saddr,panid,lqi),(proto,xtra) [CcT(ESSC.)T(i).]
# < chan,rssi,(laddr,saddr,panid,lqi),(proto,xtra) [Cct(ESSC)t(i)]
PROP_MAC_SCAN_BEACON = PROP_MAC__BEGIN + 3
PROP_MAC_15_4_LADDR = PROP_MAC__BEGIN + 4 # < [E]
PROP_MAC_15_4_SADDR = PROP_MAC__BEGIN + 5 # < [S]
@@ -136,7 +136,7 @@ class SPINEL(object):
PROP_THREAD__BEGIN = 0x50
PROP_THREAD_LEADER_ADDR = PROP_THREAD__BEGIN + 0 # < [6]
PROP_THREAD_PARENT = PROP_THREAD__BEGIN + 1 # < LADDR, SADDR [ES]
PROP_THREAD_CHILD_TABLE = PROP_THREAD__BEGIN + 2 # < [A(T(ES))]
PROP_THREAD_CHILD_TABLE = PROP_THREAD__BEGIN + 2 # < [A(t(ES))]
PROP_THREAD_LEADER_RID = PROP_THREAD__BEGIN + 3 # < [C]
PROP_THREAD_LEADER_WEIGHT = PROP_THREAD__BEGIN + 4 # < [C]
PROP_THREAD_LOCAL_LEADER_WEIGHT = PROP_THREAD__BEGIN + 5 # < [C]
@@ -144,9 +144,9 @@ class SPINEL(object):
PROP_THREAD_NETWORK_DATA_VERSION = PROP_THREAD__BEGIN + 7 # < [S]
PROP_THREAD_STABLE_NETWORK_DATA = PROP_THREAD__BEGIN + 8 # < [D]
PROP_THREAD_STABLE_NETWORK_DATA_VERSION = PROP_THREAD__BEGIN + 9 # < [S]
# < array(ipv6prefix,prefixlen,stable,flags) [A(T(6CbC))]
# < array(ipv6prefix,prefixlen,stable,flags) [A(t(6CbC))]
PROP_THREAD_ON_MESH_NETS = PROP_THREAD__BEGIN + 10
# < array(ipv6prefix,prefixlen,stable,flags) [A(T(6CbC))]
# < array(ipv6prefix,prefixlen,stable,flags) [A(t(6CbC))]
PROP_THREAD_LOCAL_ROUTES = PROP_THREAD__BEGIN + 11
PROP_THREAD_ASSISTING_PORTS = PROP_THREAD__BEGIN + 12 # < array(portn) [A(S)]
PROP_THREAD_ALLOW_LOCAL_NET_DATA_CHANGE = PROP_THREAD__BEGIN + 13 # < [b]
@@ -165,7 +165,7 @@ class SPINEL(object):
PROP_THREAD_ROUTER_DOWNGRADE_THRESHOLD = PROP_THREAD_EXT__BEGIN + 8 # < [C]
PROP_THREAD_ROUTER_SELECTION_JITTER = PROP_THREAD_EXT__BEGIN + 9 # < [C]
PROP_THREAD_PREFERRED_ROUTER_ID = PROP_THREAD_EXT__BEGIN + 10 # < [C]
PROP_THREAD_NEIGHBOR_TABLE = PROP_THREAD_EXT__BEGIN + 11 # < [A(T(ESLCcCbLL))]
PROP_THREAD_NEIGHBOR_TABLE = PROP_THREAD_EXT__BEGIN + 11 # < [A(t(ESLCcCbLL))]
PROP_THREAD_CHILD_COUNT_MAX = PROP_THREAD_EXT__BEGIN + 12 # < [C]
PROP_THREAD_EXT__END = 0x1600
@@ -181,9 +181,9 @@ class SPINEL(object):
PROP_IPV6_LL_ADDR = PROP_IPV6__BEGIN + 0 # < [6]
PROP_IPV6_ML_ADDR = PROP_IPV6__BEGIN + 1 # < [6C]
PROP_IPV6_ML_PREFIX = PROP_IPV6__BEGIN + 2 # < [6C]
# < array(ipv6addr,prefixlen,valid,preferred,flags) [A(T(6CLLC))]
# < array(ipv6addr,prefixlen,valid,preferred,flags) [A(t(6CLLC))]
PROP_IPV6_ADDRESS_TABLE = PROP_IPV6__BEGIN + 3
# < array(ipv6prefix,prefixlen,iface,flags) [A(T(6CCC))]
# < array(ipv6prefix,prefixlen,iface,flags) [A(t(6CCC))]
PROP_IPV6_ROUTE_TABLE = PROP_IPV6__BEGIN + 4
PROP_IPv6_ICMP_PING_OFFLOAD = PROP_IPV6__BEGIN + 5 # < [b]
@@ -315,8 +315,7 @@ class SPINEL(object):
PROP_CNTR_RX_ERR_OTHER = PROP_CNTR__BEGIN + 113
# The message buffer counter info
# Format: `T(SSSSSSSSSSSSSSSS)` (Read-only)
# `T(`
# Format: `SSSSSSSSSSSSSSSS` (Read-only)
# `S`, (TotalBuffers) The number of buffers in the pool.
# `S`, (FreeBuffers) The number of free message buffers.
# `S`, (6loSendMessages) The number of messages in the 6lo send queue.
@@ -332,8 +331,7 @@ class SPINEL(object):
# `S`, (ArpMessages) The number of messages in the ARP send queue.
# `S`, (ArpBuffers) The number of buffers in the ARP send queue.
# `S`, (CoapClientMessages) The number of messages in the CoAP client send queue.
# `S`, (CoapClientBuffers) The number of buffers in the CoAP client send queue.
# `)`
# `S` (CoapClientBuffers) The number of buffers in the CoAP client send queue.
PROP_MSG_BUFFER_COUNTERS = PROP_CNTR__BEGIN + 400
#=========================================