I’m looking at writing a transform for a kafka pipeline, but it’s not clear to me how the length is encoded:
From the docs: https://docs.memsql.com/memsql-pipelines/v6.8/transforms/
- Read the first eight bytes to determine the byte length of the actual message data that follows.
In the example Python code, the line is:
byte_len = struct.unpack("L", byte_len)[0]
Where "L" only means this is of type long (shouldn’t it be long long?) but does not indicate the endianess, meaning it just does whatever is native to the machine.