NHS31xx storage - Maximizing storage for equisized samples
storage: NVM Storage module

Detailed Description

The storage module allows an application to store samples all of identical size in non-volatile memories EEPROM and FLASH.

It will:

The storage module will use general purpose registers, EEPROM and FLASH to store bits of data. The usage of the three types of memories is prioritized. When a new sample is to be stored:

Writing samples always means appending them to the already written samples; it is not possible to edit the already written stream of bits. When reading out, the user can control the starting read position using a sequence number. It is automatically deduced where the corresponding sample is written, whether it is compressed, and what needs to be done to be able to return the requested sample(s).

Hardening
The storage module provides a ready-made solution to maximize storage of samples in persistent memory. It can:
  • store more than 10.000 samples without hitting the write endurance limit of both EEPROM and FLASH
  • recover state from unexpected resets
  • recover all or most of the samples in case of data corruption (this may occur when the battery is degraded and can no longer supply the minimum voltage when the load increases during a write operation in EEPROM or FLASH)
It is not possible to fully recover data under all circumstances. The storage module guarantees that only the last few samples may get lost. The number is dependent on the number of the reserved general purpose registers and the size of a sample, both of which (and more) can be tweaked using diversity settings.
Diversity
This module supports diversity settings. Some settings define the type and size of the sample. Others define the EEPROM and FLASH regions placed under control of this module. The rest of the settings control the behavior of the module. Check Diversity Settings for all diversity settings.
It is expected that each application that requires this module includes it and configures the diversity settings of the module according to its specific needs.
Memory Requirements
The storage module requires a large chunk of SRAM, called its workarea - see STORAGE_WORKAREA and STORAGE_WORKAREA_SIZE. This is used for two purposes:
  • When storing samples, and a move from EEPROM to FLASH is required, the assigned compress callback - see STORAGE_COMPRESS_CB - is given a pointer inside this SRAM memory. The output is then stored in FLASH.
  • When reading samples from FLASH, the assigned decompress callback - see STORAGE_DECOMPRESS_CB - is called as little as possible: its output is cached in the work area to speed up subsequent reads.
If two operations in your code require such a big chunk of memory, you can overlap them if they don't have to operate concurrently. Diversity setting STORAGE_WORKAREA can be used for this.
How to use the module
  1. Define the best diversity settings for your application or accept the default ones.
  2. Initialize the EEPROM driver and the storage module, in that order.
  3. Read and write samples as necessary, in any order or quantity that is required for your use case.
  4. De-initialize the storage module and the EEPROM driver, in that order.
Example
int n;
uint8_t one = 1;
uint8_t ten[10] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
Chip_EEPROM_Init(NSS_EEPROM);
Storage_Reset(false);
Storage_Write(&one, 1);
n = Storage_Read(&one, 1); /* one will remain 1 */
ASSERT(n == 1);
Storage_Write(ten, 10);
/* Read index: x
* Data stored: 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
* Write index: x
*/
n = Storage_Read(ten, 7); /* ten will become {1, 0, 1, 2, 3, 4, 5, 7, 8, 9} */
ASSERT(n == 7);
/* Read index: x
* Data stored: 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
* Write index: x
*/
n = Storage_Read(ten, 100); /* ten will become {6, 7, 8, 9, 3, 4, 5, 7, 8, 9} */
ASSERT(n == 4);
/* Read index: x
* Data stored: 1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
* Write index: x
*/
Warning
These functions are not re-entrant. Calling these functions from multiple threads or in an interrupt is highly discouraged.
The storage module requires the exclusive use of at least one register in the always-on domain (see STORAGE_FIRST_ALON_REGISTER). Under no circumstance may the reserved registers be touched from outside the storage module.
Although the storage module is able to recover after a power loss or going to Power-off, it is slow in doing so. The slow recovery time only occurs once as long as no changes to the NVM are made (e.g. by calling Storage_Write).

Modules

 Diversity Settings
 

Typedefs

typedef int(* pStorage_CompressCb_t) (int eepromByteOffset, int bitCount, void *pOut)
 
typedef int(* pStorage_DecompressCb_t) (const uint8_t *pData, int bitCount, void *pOut)
 

Functions

void Storage_Init (void)
 
void Storage_DeInit (void)
 
int Storage_GetCount (void)
 
void Storage_Reset (bool checkFlash)
 
int Storage_Write (STORAGE_TYPE *pSamples, int n)
 
bool Storage_Seek (int n)
 
int Storage_Read (STORAGE_TYPE *pSamples, int n)
 

Typedef Documentation

◆ pStorage_CompressCb_t

typedef int(* pStorage_CompressCb_t) (int eepromByteOffset, int bitCount, void *pOut)

Whenever data is about to be moved from EEPROM to FLASH, the application is notified via a callback of this prototype. It then has a chance to compress the data before it is written to FLASH. The application is in charge of:

  • reading the data from EEPROM using eepromByteOffset as starting point,
  • compressing exactly bitCount bits,
  • storing the end result in out and
  • returning the size of the data written in pOut, expressed in bits.
    Parameters
    eepromByteOffsetThe absolute offset in bytes to the EEPROM where the first sample is stored.
    bitCountAn exact total of STORAGE_BLOCK_SIZE_IN_SAMPLES samples are stored. They are packed together, i.e. without any padding bits. This argument will always equal STORAGE_UNCOMPRESSED_BLOCK_SIZE_IN_BITS.
    pOutA pointer to SRAM where the compressed data must be stored in. The buffer has a size of STORAGE_UNCOMPRESSED_BLOCK_SIZE_IN_BYTES bytes.
    Returns
    The size of the compressed data in bits. When 0 is returned, or a value bigger than or equal to STORAGE_UNCOMPRESSED_BLOCK_SIZE_IN_BITS, the uncompressed data will be stored instead, and the corresponding decompression callback of type pStorage_DecompressCb_t will not be called when reading out the data later.
    Warning
    It is not allowed to call any function of this module during the lifetime of the callback.
    See also
    STORAGE_COMPRESS_CB

Definition at line 123 of file storage.h.

◆ pStorage_DecompressCb_t

typedef int(* pStorage_DecompressCb_t) (const uint8_t *pData, int bitCount, void *pOut)

Whenever data is read from FLASH, the application is notified via a callback of this prototype. It then has a chance to decompress the data before it is used to fulfill the read request Storage_Read. The application is in charge of:

  • reading the data from FLASH using data as starting point,
  • decompressing one block of compressed data stored from that point with size bitCount,
  • storing the end result - STORAGE_BLOCK_SIZE_IN_SAMPLES samples - in out. The samples must be written packed together, i.e. without any padding bits.
    Parameters
    pDataThe absolute byte address to FLASH memory where the start of the (compressed) data block is found.
    bitCountThe size in bits of the (compressed) data block.
    pOutA pointer to SRAM where the compressed data must be stored in. The buffer has a size of STORAGE_UNCOMPRESSED_BLOCK_SIZE_IN_BYTES bytes.
    Returns
    The number of bits written to in pOut. If STORAGE_UNCOMPRESSED_BLOCK_SIZE_IN_BITS is returned, the operation is assumed to be successful and the decompressed samples are now available. Any other value indicates a decompression failure: the samples stored in that block can not be retrieved any more, and the call to Storage_Read which initiated this callback will fail.
    Warning
    It is not allowed to call any function of this module during the lifetime of the callback.
    See also
    STORAGE_DECOMPRESS_CB

Definition at line 145 of file storage.h.

Function Documentation

◆ Storage_Init()

void Storage_Init ( void  )

This function must be the first function to call in this module after going to deep power down or power-off power save mode. Not calling this function will result at best in random data being written and read, and possibly generate hard faults.

Precondition
The value written in STORAGE_FIRST_ALON_REGISTER and beyond is either exactly what was stored after leaving Storage_DeInit, or equal to 0.
EEPROM is initialized and is ready to be used.
Postcondition
When this function returns, Storage_Seek still needs to be called before being able to read samples. This is also required when reading from the start of the memory, i.e. when reading the oldest stored sample referred to as index 0.
Warning
This function can run for a long time before completion when it is forced to scan the assigned EEPROM region to recover the data that is stored in EEPROM and/or FLASH. The time spent is depending on both the assigned EEPROM region and the number of samples stored in EEPROM: the longer the region, the more time spent; the more samples stored in EEPROM, the less time spent recovering. Under worst case conditions using a system clock of 0.5 MHz, this may last more than 10 msec. This penalty only occurs under these combined conditions:
  • the IC went to power-off, losing all information stored in the register STORAGE_FIRST_ALON_REGISTER and beyond.
  • data was added to the storage module after leaving a previous power-off mode

Be sure to check out the initialization flowchart

Definition at line 1425 of file storage.c.

◆ Storage_DeInit()

void Storage_DeInit ( void  )

This function must be the last function to call in this module before going to deep power down or power-off power save mode.

Precondition
EEPROM is still initialized and ready to be used.
Postcondition
Possibly, an EEPROM flush was necessary, but that has finished when this function returns.
Warning
Loss of power before or during this call may result in loss of some or all of the newly added samples.

Definition at line 1505 of file storage.c.

◆ Storage_GetCount()

int Storage_GetCount ( void  )
Returns
The total number of samples currently stored, in EEPROM and FLASH combined.

Definition at line 1530 of file storage.c.

◆ Storage_Reset()

void Storage_Reset ( bool  checkFlash)

Resets the storage module to a pristine state.

Parameters
checkFlashThe contents in FLASH must have been erased before it can be written to. An erase operation is costly and time-consuming and is preferably avoided.
  • When true is given, all words of the FLASH memory assigned for sample storage are checked. If one checked word does not contain the erased value (0xFFFFFFFF), all FLASH memory is erased. This can possibly take up to 3 erase cycles. Aligning STORAGE_FLASH_FIRST_PAGE and STORAGE_FLASH_LAST_PAGE to sector boundaries can reduce this to the minimum of 1 erase cycle.
  • When false is given, FLASH memory is not checked and not erased.
Note
Provide true as argument for checkFlash when the intention is to store new samples afterwards.

Definition at line 1535 of file storage.c.

◆ Storage_Write()

int Storage_Write ( STORAGE_TYPE pSamples,
int  n 
)

Stores n samples.

Precondition
EEPROM is initialized
Parameters
pSamplesPointer to the start of the array where to copy the samples from. For each element of the array, only the STORAGE_BITSIZE LSBits are copied.
nThe size of the array pSamples points to, in number of STORAGE_TYPE elements.
Returns
The number of samples written. This value may be 0 or any number of samples less than or equal to n.
Note
When a value less than n is returned, at least one of these errors occurred:
  • There is insufficient storage capacity
  • Compressing of samples was necessary during the call, but that operation yielded an error.
A prior call to Storage_Seek is not required, as writing will always append the new samples.
Postcondition
A later call to Storage_DeInit is necessary to ensure the data can survive Deep power down state.
Warning
Data is not guaranteed to be stored in EEPROM or FLASH: reset can lose some of the last samples written.

Definition at line 1619 of file storage.c.

◆ Storage_Seek()

bool Storage_Seek ( int  n)

Determines which sample is read out next in a future call to Storage_Read. This call is required to be called once before calling Storage_Read one or multiple times.

Parameters
nMust be a positive number. A value of 0 indicates the oldest sample, which was written first.
Returns
true when the sought for sequence number was found; false otherwise.
Postcondition
the next call to Storage_Read will either return at least one sample - the value which was stored as the n-th sample - or fail - when less than n samples are being stored at the time of calling this function.
Note
After initialization, the default sequence number is not 0. First, a call to this function is required before Storage_Read can retrieve samples.

Definition at line 1681 of file storage.c.

◆ Storage_Read()

int Storage_Read ( STORAGE_TYPE pSamples,
int  n 
)

Reads n samples from persistent storage, starting from the sequence number set in Storage_Seek.

Precondition
EEPROM is initialized
A prior successful call to Storage_Seek is required before this function can succeed.
Note
Multiple reads can be issued after calling Storage_Seek once, each time fetching samples in sequence.
Parameters
[out]pSamples: Pointer to an array of
elements, where the read samples are copied to. Upon successful completion, each element will contain one sample, where only the STORAGE_BITSIZE LSBits are used per element; the remainder MSBits are set to 0.
n: The size of the array samples points to, in number of STORAGE_TYPE elements.
Returns
The number of samples read. This value may be 0 or any number of samples less than n. The remainder of the elements with a higher index may have been written to, but must be ignored.
Note
When a value less than n is returned, at least one of these errors occurred:
  • There was no prior successful call to Storage_Seek
  • Decompressing of samples was necessary during the call, but that operation yielded an error.
  • There are no more samples stored.

Definition at line 1750 of file storage.c.