NHS31xx storage - Maximizing storage for equisized samples
storage.c File Reference

Go to the source code of this file.

Detailed Description

Maintaining state
It is a crucial feature of this module to be able to resume operation at all times. Between de-initialization and re-initialization anything can happen, and no volatile memory can be trusted. Still, speed of operation is necessary. To be able to restore the exact state - done in Storage_Init - three different recovery structures are used:
  • RecoverInfo_t - stored in the general purpose registers
  • Hint_t - stored on a fixed location in EEPROM
  • Marker_t - stored on a variable location, immediately after the last written sample in EEPROM
After recovery, state is maintained in SRAM using the structure Storage_Instance_t, and in Storage_DeInit at least two of the three recovery structures are updated.
The recovery structures are linked to each other as explained below.
  • RecoverInfo_t This data is stored in the designated ALON register STORAGE_FIRST_ALON_REGISTER and is by far the fastest and easiest way to recover the current state. It is always tried first. If its data is non-zero, the information is trusted and used.
  • Hint_t This data is stored in the last page of the assigned EEPROM region, and also enables recovery of the current state in a fast way. However, this structure may be out of date as it may not be updated at every change. The reason for not updating is to ensure the maximum number of writes (endurance) is never reached. However, this data will always accurately tell
    • whether samples are stored or not
    • how much flash is occupied by data storage
    It also provides a possible location to Marker_t, which may be outdated.
  • Marker_t This data is always stored just after the last sample written in EEPROM. Its precise location is not known, as it is progressing together with the data. RecoverInfo_t points to the start of this structure in the assigned EEPROM region; Hint_t may point to it; if both are failing a full slow search is performed in the assigned EEPROM region.

When a full backward search is performed during data recovery, we rely on the length of the marker to eliminate false positives, i.e. to ensure no bit sequence exists that looks like a valid marker but are in reality one or more samples stored in EEPROM. The comments near the code in FindMarker explain the length and value of the marker avoid finding false positives. Yet, although extremely improbable, there is still a very tiny possibility that real-life data being stored in EEPROM results in such a false positive. Even then, most applications will never have the need to perform a full backward search, evading this problem. If your application is more likely to hit this, due to the use case being sufficiently different, and due to the samples being sufficiently similar, this module can be adapted to erase the full EEPROM after moving all data to FLASH - see MoveSamplesFromEepromToFlash. During our continuous tests using randomized data of different lengths, we have never encountered this problem and therefore decided not to suffer the extra cost (time and power consumption) of implementing this.

Recovering
In Storage_DeInit, at least the ALON register (RecoverInfo_t) and the EEPROM marker (Marker_t) are updated. Assuming only one sample was added, the EEPROM contents would then change as visually depicted below:
Note
the pictures below use this legend:
  • aaa, bbb, ... fff: samples, with bitsize STORAGE_BITSIZE
  • AA: ALON register, formatted according to RecoverInfo_t
  • HH: hint, formatted according to Hint_t
  • MMMMMMM: marker, formatted according to Marker_t
  • BB: block info, of size FLASH_DATA_HEADER_SIZE
  • xxxx: block bytes, the (compressed) data bytes moved from EEPROM and now stored in FLASH
EEPROM EEPROM
with samples a..d with samples a..e
+---------+ +---------+
| aaabbbc | | aaabbbc |
| ccdddMM | After calling | ccdddee |
| MMMMM | Storage_Write | eMMMMMM |
| | with n=1: | MM |
| | ------------------> | |
| HH | | HH |
+---------+ +---------+

The ALON register will then point to the new location of the marker; while the hint will not be updated and will for sure contain faulty information.

dot_inline_dotgraph_1.png

When power gets lost, the ALON register contents is reset to 0, and will then point to a faulty location as well.

dot_inline_dotgraph_2.png

Only the marker will then still be correct, which can be recovered only through a full backward search. When that is done, after leaving Storage_DeInit, all three recovery structures will be correctly linked.

dot_inline_dotgraph_3.png
Data corruption
When the battery is degrading, two scenarios can cause a corruption:
  • the supplied voltage drops too much during a flash operation, hanging the IC or causing a reset. This can corrupt the FLASH portion being written to.
  • the supplied voltage drops too much during an EEPROM operation, hanging the IC or causing a reset. This can corrupt the EEPROM row being written to.
The storage module can be fully resilient against this: no data present in the non-volatile memories EEPROM and FLASH is lost and all samples can still be read out, with one exception: the samples written after the last update to the hint structure cannot be recovered, as no duplicate information is yet available.

After a hard reset, the general purpose registers are reset to 0, and the samples temporarily stored there are also lost. This brings the total number of minimum lost samples to STORAGE_WRITE_RECOVERY_EVERY_X_SAMPLES + STORAGE_SAMPLE_ALON_CACHE_COUNT - both are diversity settings the application can set.

Using EEPROM and FLASH
Both EEPROM and FLASH are used to store data - in the form of equisized samples. Since writing to EEPROM is cheaper, faster and less complicated than writing to FLASH, it is the preferred storage medium. Whenever the size of all the samples stored in EEPROM is large enough - see STORAGE_BLOCK_SIZE_IN_SAMPLES - all that data is moved in one operation to FLASH. EEPROM is then completely empty, and new samples are then in EEPROM again.

The memory content changes are visually depicted below. If writing a new sample would increase the size to equal to or higher than STORAGE_BLOCK_SIZE_IN_SAMPLES, first the EEPROM contents are moved, then the new sample is written.

EEPROM EEPROM
with samples a..e with sample f only
+---------+ +---------+
| aaabbbc | | fffMMMM |
| cc..... | | MMM |
| ....... | | |
| ....ddd | ----------------------> | |
| eeeMMMM | | |
| MMM HH | | HH |
+---------+ +---------+
FLASH FLASH
with one block moved from EEPROM with two blocks moved from EEPROM
+---------+ +---------+
| BBxxxxx | | BBxxxxx |
| xxxxxxx | | xxxxxxx |
| x | ----------------------> | xBBxxxx |
| | | xxxxx |
| | | |
| | | |
+---------+ +---------+
Caching data
To reduce the number of EEPROM flushes, data is initially not stored in non-volatile memories EEPROM and FLASH. Instead, the reserved general purpose registers are used to cache the data. The value is preserved when entering and leaving a deep power down, but any inadvertent reset clears the registers, losing the data. This is not a disadvantage, since the cached data will be moved to non-volatile memory before writing the hint The benefit is a reduction of the required EEPROM program operations. This helps in avoiding the write endurance limit even when storing massive amounts of data, and also reduces the stress on the battery when it nears end-of-life.

Initializing
The implementation of Storage_Init tries to recover the complete state as fast as possible. This can be done when either the recovery info or the hint information points to a valid marker structure. When a slow search is required, the initialization function make sure a next call will be fast.
Storage_Init implements this flowchart:
dot_inline_dotgraph_4.png

Definition in file storage.c.

Data Structures

struct  RecoverInfo_t
 
struct  Hint_t
 
struct  Marker_t
 
struct  Storage_Instance_t
 

Macros

#define EEPROM_ABSOLUTE_FIRST_BYTE_OFFSET   (STORAGE_EEPROM_FIRST_ROW * EEPROM_ROW_SIZE)
 
#define EEPROM_ABSOLUTE_LAST_BYTE_OFFSET   (((STORAGE_EEPROM_LAST_ROW + 1) * EEPROM_ROW_SIZE) - 1)
 
#define FLASH_CURSOR_TO_BYTE_ADDRESS(flashByteCursor)   ((uint8_t *)(FLASH_START + (STORAGE_FLASH_FIRST_PAGE * FLASH_PAGE_SIZE) + (flashByteCursor)))
 
#define FLASH_CURSOR_TO_PAGE(flashByteCursor)   (STORAGE_FLASH_FIRST_PAGE + ((flashByteCursor) / FLASH_PAGE_SIZE))
 
#define FLASH_PAGE_TO_ADDRESS(type, flashPage)   ((type)(FLASH_START + ((flashPage) * FLASH_PAGE_SIZE)))
 
#define FLASH_FIRST_BYTE_ADDRESS   FLASH_PAGE_TO_ADDRESS(uint8_t *, STORAGE_FLASH_FIRST_PAGE)
 
#define FLASH_LAST_BYTE_ADDRESS   (FLASH_PAGE_TO_ADDRESS(uint8_t *, STORAGE_FLASH_LAST_PAGE + 1) - 1)
 
#define FLASH_FIRST_WORD_ADDRESS   FLASH_PAGE_TO_ADDRESS(uint32_t *, STORAGE_FLASH_FIRST_PAGE)
 
#define FLASH_LAST_WORD_ADDRESS   (FLASH_PAGE_TO_ADDRESS(uint32_t *, STORAGE_FLASH_LAST_PAGE + 1) - 1)
 
#define FLASH_DATA_HEADER_SIZE   STORAGE_BLOCK_HEADER_SIZE
 
#define FLASH_BLOCK_SIZE(bitCount)   (4 * STORAGE_IDIVUP((bitCount) + FLASH_DATA_HEADER_SIZE * 8, 32))
 
#define MARKER_HEADER   ((int)0x0000FFFF)
 
#define MARKER_FOOTER   ((int)0x7FFFFFFF)
 
#define MARKER_CURSOR_ZERO_MASK   0xFFFF8003
 
#define STORAGE_FLASH_FIRST_PAGE   sStorageFlashFirstPage
 
#define FIRST_BITS_OF_CACHE_SIZE   13
 
#define SIZE_OF_HINT   4
 
#define INVERSE_HINT_ABSOLUTE_BYTE_OFFSET   ((EEPROM_ABSOLUTE_LAST_BYTE_OFFSET + 1 - SIZE_OF_HINT))
 
#define HINT_ABSOLUTE_BYTE_OFFSET   (INVERSE_HINT_ABSOLUTE_BYTE_OFFSET - SIZE_OF_HINT)
 
#define SIZE_OF_DUPLICATE_DATA   64
 
#define DUPLICATE_DATA_ABSOLUTE_BYTE_OFFSET   (EEPROM_ABSOLUTE_LAST_BYTE_OFFSET + 1 - (2 * EEPROM_ROW_SIZE))
 
#define SIZE_OF_MARKER   12
 
#define EEPROM_OVERHEAD_IN_BITS   ((SIZE_OF_MARKER + 2 * EEPROM_ROW_SIZE) * 8)
 

Enumerations

enum  LOCATION_T {
  LOCATION_UNKNOWN,
  LOCATION_CACHE,
  LOCATION_EEPROM,
  LOCATION_FLASH
}
 

Functions

int STORAGE_COMPRESS_CB (int eepromByteOffset, int bitCount, void *pOut)
 
int STORAGE_DECOMPRESS_CB (const uint8_t *pData, int bitCount, void *pOut)
 
int Storage_DummyCompressCb (int eepromByteOffset, int bitCount, void *pOut)
 
int Storage_DummyDecompressCb (const uint8_t *pData, int bitCount, void *pOut)
 
static void ResetInstance (void)
 
static void ShiftAlignedData (uint8_t *pTo, const uint8_t *pFrom, const int bitAlignment, const int bitCount)
 
static void ShiftUnalignedData (uint8_t *pTo, const uint8_t *pFrom, const int bitAlignment, const int bitCount)
 
static bool CacheSample (const STORAGE_TYPE *pSample)
 
static bool GetCachedSample (const int n, void *pData)
 
static void WriteToEeprom (const int bitCursor, const void *pData, const int bitCount)
 
static void ReadFromEeprom (const unsigned int bitCursor, void *pData, const int bitCount)
 
static unsigned int FindMarker (Marker_t *pMarker)
 
static int GetEepromCount (void)
 
static int GetFlashCount (void)
 
static int StoreSamplesInEeprom (const STORAGE_TYPE *pSamples, int n)
 
static bool MoveSamplesFromEepromToFlash (void)
 
static int ReadAndCacheSamplesFromFlash (int readCursor)
 
static bool ValidateRecoverInfo (void)
 
static bool ValidateMarker (const Marker_t *pMarker, int expectedFlashByteCursor)
 
static bool ValidateHint (const Hint_t *pHint)
 
static void WriteHint (void)
 
static void WriteMarker (void)
 
void Storage_Init (void)
 
void Storage_DeInit (void)
 
int Storage_GetCount (void)
 
void Storage_Reset (bool checkFlash)
 
int Storage_Write (STORAGE_TYPE *samples, int n)
 
bool Storage_Seek (int n)
 
int Storage_Read (STORAGE_TYPE *samples, int n)
 

Variables

const int _etext
 
const int _data
 
const int _edata
 
static int sStorageFlashFirstPage
 
int checkSizeOfSampleType [(sizeof(STORAGE_TYPE) *8< STORAGE_BITSIZE) ? -1 :1]
 
int checkSizeOfHint [(SIZE_OF_HINT==sizeof(Hint_t)) ? 1 :-1]
 
int checkSizeOfMarker [(SIZE_OF_MARKER==sizeof(Marker_t)) ? 1 :-1]
 
int checkSizeOfRecoverInfo [sizeof(RecoverInfo_t)==4 ? 1 :-1]
 
static Storage_Instance_t sInstance
 
static uint8_t sCache [4 *(5 - STORAGE_FIRST_ALON_REGISTER)]
 
static RecoverInfo_tspRecoverInfo
 
static bool sEepromBitCursorChanged = false
 
uint8_t sStorage_Workarea [STORAGE_WORKAREA_SIZE]
 
uint8_t STORAGE_WORKAREA [STORAGE_WORKAREA_SIZE]
 

Data Structure Documentation

◆ RecoverInfo_t

struct RecoverInfo_t

An instance of this structure is stored in the designated register of the ALON domain, and points to where to find Marker_t. This structure is stored in ALON and allows for the fastest initialization. If all these values are 0, either no logging is ongoing, or the battery has gone empty. To discover which case holds true, first Hint_t is checked; if that fails too, the EEPROM is checked backwards for the location of a stored instance of type Marker_t (which is time consuming). Once found, the correct values for this structure can be reconstructed.

Definition at line 422 of file storage.c.

Data Fields
unsigned int eepromBitCursor: 15
See also
Storage_Instance_t.eepromBitCursor
unsigned int sampleCacheCount: 4

The number of samples that are cached in the general purpose registers. These are stored starting from firstBitsOfCache up to the last general purpose register. A value of 0 indicates no samples are cached.

unsigned int firstBitsOfCache: FIRST_BITS_OF_CACHE_SIZE

A filler. It allows to perform a sanity check at compile time on the size of RecoverInfo_s.

◆ Hint_t

struct Hint_t

An instance of this structure is stored at the fixed location HINT_ABSOLUTE_BYTE_OFFSET, and points to where to find Marker_t. If all the information contained is correct (which may not be the case) it still allows for a fast initialization.

  • eepromBitCursor may contain correct information. If not, the EEPROM is checked backwards for the location of a stored instance of type Marker_t (which is time consuming) and this hint structure is adapted (so a next initialization under the same circumstances is done quickly).
  • The Hint_t instance also contains a duplicate reference to flashByteCursor. This will be used when the marker is no longer available or is corrupt.

Definition at line 473 of file storage.c.

Data Fields
uint16_t eepromBitCursor
See also
Storage_Instance_t.eepromBitCursor
uint16_t flashByteCursor
See also
Storage_Instance_t.flashByteCursor

◆ Marker_t

struct Marker_t

An instance of this structure is stored in EEPROM, right after the end of the last sample in EEPROM. It is used to re-create sInstance upon initialization. Since a number of false candidates may be proposed, it contains a header and a footer to have a near 100% chance of detecting erroneous information.

Definition at line 487 of file storage.c.

Data Fields
const int header

Must equal MARKER_HEADER, or flashByteCursor is not valid.

int flashByteCursor
See also
Storage_Instance_t.flashByteCursor
Note
Usage
const int footer

Must equal MARKER_FOOTER, or flashByteCursor is not valid.

◆ Storage_Instance_t

struct Storage_Instance_t

Stores all meta data to perform the requested reads and writes in FLASH and EEPROM. Upon de-initialization, an updated instance of Marker_t is updated and stored in EEPROM, and an updated instance of RecoverInfo_t is stored in the ALON domain, using the information in this structure.

Definition at line 517 of file storage.c.

Data Fields
int eepromBitCursor

The position of the first bit where new data can be stored in EEPROM, relative to STORAGE_EEPROM_FIRST_ROW. This doubles up as the start position of a stored instance of type Marker_t in EEPROM. There are at most EEPROM_NR_OF_RW_ROWS rows of EEPROM that can be used for storage, thus an EEPROM bit cursor always has values less than 2**15: 15 bits are required to store this information.

int flashByteCursor

The position of the first byte where new data can be stored in FLASH, relative to STORAGE_FLASH_FIRST_PAGE.

Precondition
Must be 32-bit word-aligned: multiple FLASH writes in the same page must occur at word boundaries. See FLASH_BLOCK_SIZE
Note
There are at most (FLASH_NR_OF_RW_SECTORS * FLASH_PAGES_PER_SECTOR - program size) pages of FLASH that can be used for storage, thus a FLASH byte cursor always has values less than 2**15; and due to the boundary requirement, the 2 LSBits must be 0 as well: 13 bits are thus required to store this information. Masking with MARKER_CURSOR_ZERO_MASK must thus always yield a 0 value.
LOCATION_T readLocation

Indicates the type of memory to readCursor targets. It also defines the type of readCursor: a byte cursor or a bit cursor.

int readCursor
  • If readLocation equals LOCATION_FLASH: the offset in bytes relative to FLASH_FIRST_BYTE_ADDRESS where to read next in FLASH. This points to the two-byte header that precedes a (compressed) data block, stored in FLASH.
  • If readLocation equals LOCATION_EEPROM: the offset in bits relative to EEPROM_ABSOLUTE_FIRST_BYTE_OFFSET where to read next in EEPROM. This points to the position of the LSBit of the next sample to read.
  • If readLocation equals LOCATION_CACHE: a relative index. The index of the sample that is stored in the cache. 0 indicates the oldest sample still present in the cache.
    Note
    A negative value indicates nothing can be read. A call to Storage_Seek is then required.
int readSequence

A sequence number to help identify where the next sample, which must be read out, is located.

  • In case of reading from FLASH, where (compressed) blocks of samples are stored, this is the absolute sequence number of the first sample in that block after decompressing.
  • In case of reading from EEPROM, this is the absolute sequence number of the sample readCursor points to.
  • In case of reading from sCache, this is the absolute sequence number of the sample readCursor points to. The very first sample written via Storage_Write has sequence number 0.
    Note
    A negative value indicates nothing can be read. A call to Storage_Seek is then required.
int targetSequence

The sequence number that was requested in the last call to Storage_Seek.

int cachedBlockOffset

Determines what contents are available in STORAGE_WORKAREA.

  • The flash byte offset relative to FLASH_FIRST_BYTE_ADDRESS where the header preceding the (compressed) data block can be found; these contents are decompressed and the samples, packed without padding bits, are available from the start of STORAGE_WORKAREA.
  • -1 if STORAGE_WORKAREA does not contain samples after decompressing a data block stored in FLASH, or if STORAGE_WORKAREA is used to compress data.

Macro Definition Documentation

◆ EEPROM_ABSOLUTE_FIRST_BYTE_OFFSET

#define EEPROM_ABSOLUTE_FIRST_BYTE_OFFSET   (STORAGE_EEPROM_FIRST_ROW * EEPROM_ROW_SIZE)

The absolute offset to the very first byte of the assigned EEPROM region.

Note
Unless explicitly specified, all bit and byte offsets referring to the EEPROM region in the code are relative to this value.

Definition at line 310 of file storage.c.

Referenced by FindMarker(), MoveSamplesFromEepromToFlash(), ReadFromEeprom(), Storage_Init(), ValidateHint(), WriteHint(), and WriteToEeprom().

◆ EEPROM_ABSOLUTE_LAST_BYTE_OFFSET

#define EEPROM_ABSOLUTE_LAST_BYTE_OFFSET   (((STORAGE_EEPROM_LAST_ROW + 1) * EEPROM_ROW_SIZE) - 1)

The absolute offset to the very last byte of the assigned EEPROM region.

Definition at line 314 of file storage.c.

Referenced by FindMarker().

◆ FLASH_CURSOR_TO_BYTE_ADDRESS

#define FLASH_CURSOR_TO_BYTE_ADDRESS (   flashByteCursor)    ((uint8_t *)(FLASH_START + (STORAGE_FLASH_FIRST_PAGE * FLASH_PAGE_SIZE) + (flashByteCursor)))

Translates a flash byte cursor relative to assigned FLASH region to a byte address.

Definition at line 317 of file storage.c.

Referenced by GetFlashCount(), MoveSamplesFromEepromToFlash(), ReadAndCacheSamplesFromFlash(), Storage_Seek(), and ValidateMarker().

◆ FLASH_CURSOR_TO_PAGE

#define FLASH_CURSOR_TO_PAGE (   flashByteCursor)    (STORAGE_FLASH_FIRST_PAGE + ((flashByteCursor) / FLASH_PAGE_SIZE))

Translates a FLASH byte cursor relative to assigned FLASH region to the number of the page the cursor refers to.

Definition at line 321 of file storage.c.

Referenced by MoveSamplesFromEepromToFlash().

◆ FLASH_PAGE_TO_ADDRESS

#define FLASH_PAGE_TO_ADDRESS (   type,
  flashPage 
)    ((type)(FLASH_START + ((flashPage) * FLASH_PAGE_SIZE)))

Translates a FLASH page number to the start address of that page.

Definition at line 324 of file storage.c.

◆ FLASH_FIRST_BYTE_ADDRESS

#define FLASH_FIRST_BYTE_ADDRESS   FLASH_PAGE_TO_ADDRESS(uint8_t *, STORAGE_FLASH_FIRST_PAGE)

The very first byte address of the assigned FLASH region.

Definition at line 327 of file storage.c.

◆ FLASH_LAST_BYTE_ADDRESS

#define FLASH_LAST_BYTE_ADDRESS   (FLASH_PAGE_TO_ADDRESS(uint8_t *, STORAGE_FLASH_LAST_PAGE + 1) - 1)

The very last byte address of the assigned FLASH region.

Definition at line 330 of file storage.c.

Referenced by MoveSamplesFromEepromToFlash(), and ValidateMarker().

◆ FLASH_FIRST_WORD_ADDRESS

#define FLASH_FIRST_WORD_ADDRESS   FLASH_PAGE_TO_ADDRESS(uint32_t *, STORAGE_FLASH_FIRST_PAGE)

The very first 32-bit word address of the assigned FLASH region.

Definition at line 333 of file storage.c.

Referenced by Storage_Reset().

◆ FLASH_LAST_WORD_ADDRESS

#define FLASH_LAST_WORD_ADDRESS   (FLASH_PAGE_TO_ADDRESS(uint32_t *, STORAGE_FLASH_LAST_PAGE + 1) - 1)

The very last WORD address of the assigned FLASH region.

Definition at line 336 of file storage.c.

Referenced by Storage_Reset().

◆ FLASH_DATA_HEADER_SIZE

#define FLASH_DATA_HEADER_SIZE   STORAGE_BLOCK_HEADER_SIZE

The size in bytes of the meta data stored just in front of the (compressed) data block in FLASH. The LSBit of the first byte after these header size contains the start of the (compressed) data block. This header is used to give the decompress callback the correct arguments, and to deduce where to find the next block.

Note
Either this size is a value greater than 0 but less than STORAGE_UNCOMPRESSED_BLOCK_SIZE_IN_BITS, which indicates the contents have been compressed; either this size is equal to STORAGE_UNCOMPRESSED_BLOCK_SIZE_IN_BITS which indicates no compression/decompression algorithm was set or compression yielded bad results on this block and therefore the block was stored uncompressed.
See also
FLASH_BLOCK_SIZE

Definition at line 349 of file storage.c.

Referenced by MoveSamplesFromEepromToFlash(), and ReadAndCacheSamplesFromFlash().

◆ FLASH_BLOCK_SIZE

#define FLASH_BLOCK_SIZE (   bitCount)    (4 * STORAGE_IDIVUP((bitCount) + FLASH_DATA_HEADER_SIZE * 8, 32))

Given the header value of a (compressed) data block in FLASH, calculates the size in bytes of the complete block, including FLASH_DATA_HEADER_SIZE. The start of a new block must always be stored on a 32-bit word boundary: this allows to continue writing the same page without erasing. As a consequence, up to 31 padding bits after the end of previous (compressed) data block are lost.

Parameters
bitCountThe size in bits of the (compressed) data block, excluding FLASH_DATA_HEADER_SIZE.
Note
  • From the FLASH specification: It is possible to write only a sub-set of the page (Y words), but the maximum number of times that a page can be written to, before an erase must be performed, is 16. WARNING: Writing a page more than 16 times without erasing may result in corrupting the page’s contents, due to write disturb (an intrinsic mechanism to the programming mechanism of the C14EFLASH)
  • From the NVMC datasheet: Due to the presence of ECC, it is not allowed to modify additional bits inside a memory word where some bits have already been programmed: the resulting ECC code in memory would then be the AND of the codes for the previous and new value written, and will most likely be inconsistent with the resulting data, potentially resulting in unwanted or missing bit corrections, or spurious error conditions. [C-NODPG]

Definition at line 373 of file storage.c.

Referenced by GetFlashCount(), MoveSamplesFromEepromToFlash(), ReadAndCacheSamplesFromFlash(), and Storage_Seek().

◆ MARKER_HEADER

#define MARKER_HEADER   ((int)0x0000FFFF)

The first of two special values that are used in Marker_t to be able to reconstruct the EEPROM and FLASH bit cursor in case the battery has died - and thus the register value has been reset to zero.

Note
These values are not chosen randomly: the code using them - FindMarker - is depending on their specific bit values.

Definition at line 381 of file storage.c.

Referenced by ValidateMarker(), and WriteMarker().

◆ MARKER_FOOTER

#define MARKER_FOOTER   ((int)0x7FFFFFFF)

The second special value.

See also
MARKER_HEADER

Definition at line 382 of file storage.c.

Referenced by ValidateMarker(), and WriteMarker().

◆ MARKER_CURSOR_ZERO_MASK

#define MARKER_CURSOR_ZERO_MASK   0xFFFF8003

The mask to use to zero out all possible 1 bits in Marker_t.flashByteCursor

Definition at line 385 of file storage.c.

Referenced by ValidateMarker().

◆ STORAGE_FLASH_FIRST_PAGE

#define STORAGE_FLASH_FIRST_PAGE   sStorageFlashFirstPage

Definition at line 396 of file storage.c.

Referenced by MoveSamplesFromEepromToFlash(), and Storage_Reset().

◆ FIRST_BITS_OF_CACHE_SIZE

#define FIRST_BITS_OF_CACHE_SIZE   13

The size of the first bits of the cache, in the same general purpose register where RecoverInfo_t is stored.

Definition at line 412 of file storage.c.

Referenced by CacheSample(), and GetCachedSample().

◆ SIZE_OF_HINT

#define SIZE_OF_HINT   4

Byte size of Hint_t. Checked at compile time using checkSizeOfHint.

Definition at line 437 of file storage.c.

◆ INVERSE_HINT_ABSOLUTE_BYTE_OFFSET

#define INVERSE_HINT_ABSOLUTE_BYTE_OFFSET   ((EEPROM_ABSOLUTE_LAST_BYTE_OFFSET + 1 - SIZE_OF_HINT))

The absolute offset to a copy of the hint structure, to validate its contents.

Definition at line 440 of file storage.c.

Referenced by ValidateHint(), and WriteHint().

◆ HINT_ABSOLUTE_BYTE_OFFSET

#define HINT_ABSOLUTE_BYTE_OFFSET   (INVERSE_HINT_ABSOLUTE_BYTE_OFFSET - SIZE_OF_HINT)

The absolute offset to Hint_t at the end of the assigned EEPROM region.

Definition at line 443 of file storage.c.

Referenced by Storage_DeInit(), Storage_Init(), Storage_Reset(), and WriteHint().

◆ SIZE_OF_DUPLICATE_DATA

#define SIZE_OF_DUPLICATE_DATA   64

Byte size of the duplicate information.

Definition at line 452 of file storage.c.

Referenced by Storage_Init(), and WriteHint().

◆ DUPLICATE_DATA_ABSOLUTE_BYTE_OFFSET

#define DUPLICATE_DATA_ABSOLUTE_BYTE_OFFSET   (EEPROM_ABSOLUTE_LAST_BYTE_OFFSET + 1 - (2 * EEPROM_ROW_SIZE))

The absolute offset to the recovery information at the end of the assigned EEPROM region.

Definition at line 459 of file storage.c.

Referenced by Storage_Init(), and WriteHint().

◆ SIZE_OF_MARKER

#define SIZE_OF_MARKER   12

The total number of bytes in EEPROM consumed by meta-data, to be able to keep track of what is stored in FLASH and EEPROM, even after a power-off. Apart from Marker_t, there is an additional bit that marks whether the storage in FLASH and EEPROM is empty or at least one sample is stored in EEPROM -

See also
Hint_t

Definition at line 510 of file storage.c.

◆ EEPROM_OVERHEAD_IN_BITS

#define EEPROM_OVERHEAD_IN_BITS   ((SIZE_OF_MARKER + 2 * EEPROM_ROW_SIZE) * 8)

The assigned EEPROM region cannot be used fully: there must be always room for Marker_t and the last couple of bytes are used to Hint_t. To avoid corruption in both the marker and the hint, the hint is placed in a separate row. The total overhead is summed up here.

Definition at line 600 of file storage.c.

Referenced by FindMarker().

Enumeration Type Documentation

◆ LOCATION_T

enum LOCATION_T
See also
Storage_Instance_t.readLocation
Enumerator
LOCATION_UNKNOWN 

A successful call to Storage_Seek is required.

LOCATION_CACHE 

Memory in the assigned general purpose registers is targeted.

LOCATION_EEPROM 

Memory in the assigned EEPROM region is targeted.

LOCATION_FLASH 

Memory in the assigned FLASH region is targeted.

Definition at line 402 of file storage.c.

Function Documentation

◆ STORAGE_COMPRESS_CB()

int STORAGE_COMPRESS_CB ( int  eepromByteOffset,
int  bitCount,
void *  pOut 
)

◆ STORAGE_DECOMPRESS_CB()

int STORAGE_DECOMPRESS_CB ( const uint8_t *  pData,
int  bitCount,
void *  pOut 
)

◆ Storage_DummyCompressCb()

int Storage_DummyCompressCb ( int  eepromByteOffset,
int  bitCount,
void *  pOut 
)

This is a dummy implementation to provoke fallback behavior: a plain copy from EEPROM to FLASH. It is only used when STORAGE_COMPRESS_CB is not overridden in an application specific app_sel.h header file

See also
pStorage_CompressCb_t
STORAGE_COMPRESS_CB
Parameters
eepromByteOffsetunused
bitCountunused
pOutunused
Returns
0

Definition at line 681 of file storage.c.

◆ Storage_DummyDecompressCb()

int Storage_DummyDecompressCb ( const uint8_t *  pData,
int  bitCount,
void *  pOut 
)

This is a dummy implementation to provoke a failure; it should normally never be called, since the dummy compression implementation will provoke the fallback behavior: a plain copy from EEPROM to FLASH, and vice-versa while reading. It is only used when STORAGE_COMPRESS_CB is not overridden in an application specific app_sel.h header file

See also
pStorage_DecompressCb_t
STORAGE_DECOMPRESS_CB
Parameters
pDataunused
bitCountunused
pOutunused
Returns
0

Definition at line 696 of file storage.c.

◆ ResetInstance()

static void ResetInstance ( void  )
static

Re-initializes sInstance with default values.

Note
No EEPROM reads or writes, no FLASH reads or writes are done.
Postcondition
sBitCursorChanged is set when samples have become inaccessible.

Definition at line 735 of file storage.c.

Referenced by Storage_Init(), and Storage_Reset().

◆ ShiftAlignedData()

static void ShiftAlignedData ( uint8_t *  pTo,
const uint8_t *  pFrom,
const int  bitAlignment,
const int  bitCount 
)
static

Copies a number of bits of byte aligned data to a buffer, non-byte aligned.

Parameters
pToThe location to copy to. A number of LSBits of the first byte are not touched.
pFromThe location to copy from. The first bit to copy will become the LSBit of the first byte in this buffer.
bitAlignmentThe number of bits to disregard in to. Must be less than 8.
bitCountThe number of bits to copy.
Precondition
STORAGE_IDIVUP(bitCount, 8)
bytes must be available in from.
Postcondition
STORAGE_IDIVUP(bitCount + bitAlignment, 8)
bytes will be written to to.
Note
The first bitAlignment bits of the first byte written to are not touched.
For example, use this function to move bits in this way:
  • With - indicating not touched or don't care,
  • bitAlignment equal to 5, and
  • bitCount equal to 14:
8765 4321 --fe dcba is copied as 321- ---- cba8 7654 0000 0fed

Definition at line 762 of file storage.c.

Referenced by CacheSample(), and WriteToEeprom().

◆ ShiftUnalignedData()

static void ShiftUnalignedData ( uint8_t *  pTo,
const uint8_t *  pFrom,
const int  bitAlignment,
const int  bitCount 
)
static

Copies a number of bits of non-byte aligned data to a buffer, byte aligned.

Parameters
pToThe location to copy to. The first bit to copy will become the LSBit of the first byte in this buffer.
pFromThe location to copy from. A number of LSBits are disregarded.
bitAlignmentThe number of bits to disregard in from. Must be less than 8.
bitCountThe number of bits to copy.
Precondition
STORAGE_IDIVUP(bitCount + bitAlignment, 8)
bytes must be available in from.
Postcondition
STORAGE_IDIVUP(bitCount, 8)
bytes will be written to to.
Note
The remainder bits in to after bitCount bits is set to 0.
For example, use this function to move bits in this way:
  • With - indicating not touched or don't care,
  • bitAlignment equal to 5, and
  • bitCount equal to 14:
321- ---- cba8 7654 ---- -fed is copied as 8765 4321 00fe dcba

Definition at line 817 of file storage.c.

Referenced by GetCachedSample(), ReadFromEeprom(), and Storage_Read().

◆ CacheSample()

static bool CacheSample ( const STORAGE_TYPE pSample)
static

Stores data in sCache.

Parameters
pSampleMay not be NULL. The data to cache.
Returns
true when the sample was copied; false when it did not fit and no changes were made.

Definition at line 868 of file storage.c.

Referenced by Storage_Write().

◆ GetCachedSample()

static bool GetCachedSample ( const int  n,
void *  pData 
)
static

Retrieves a sample cached in an earlier call to CacheSample.

Parameters
nA relative index. Determines the sample to copy. A value of 0 indicates the oldest sample that is present in sCache.
pDataMay not be NULL. The sample is copied to the array pointed to.
Returns
true if a sample was copied; false if less than n samples are stored in sCache

Definition at line 890 of file storage.c.

Referenced by Storage_Read(), and Storage_Write().

◆ WriteToEeprom()

static void WriteToEeprom ( const int  bitCursor,
const void *  pData,
const int  bitCount 
)
static
Precondition
EEPROM is initialized
Enough free space must be available in EEPROM starting from bitCursor
Parameters
bitCursorMust be positive. The first bit where to start writing.
pDataMay not be NULL.
bitCountMust be strict positive.
Postcondition
sInstance is not touched.

Definition at line 912 of file storage.c.

Referenced by MoveSamplesFromEepromToFlash(), Storage_Reset(), StoreSamplesInEeprom(), and WriteMarker().

◆ ReadFromEeprom()

static void ReadFromEeprom ( const unsigned int  bitCursor,
void *  pData,
const int  bitCount 
)
static
Precondition
EEPROM is initialized
Enough bits to read are available in EEPROM starting from bitCursor
Parameters
bitCursorMust be strict positive. The bit position in EEPROM to start reading: 0 means to start reading from the very first EEPROM bit assigned for bit storage, and so on.
pDataMay not be NULL. All bits are copied in the array pointed to.
bitCountMust be strict positive. When not a multiple of 8, the remainder MSBits of the last byte are set to 0.
Postcondition
sInstance is not touched.

Definition at line 942 of file storage.c.

Referenced by FindMarker(), Storage_Init(), and Storage_Read().

◆ FindMarker()

static unsigned int FindMarker ( Marker_t pMarker)
static

Search backwards in the assigned EEPROM region, looking for a valid marker.

Parameters
[out]pMarker: Where to copy the found marker data to. If 0 is returned, this will contain an invalid marker.
Returns
The position of the first bit of a valid stored instance of type Marker_t in EEPROM, relative to STORAGE_EEPROM_FIRST_ROW. 0 if the marker could not be found.

Definition at line 966 of file storage.c.

Referenced by Storage_Init().

◆ GetEepromCount()

static int GetEepromCount ( void  )
static
Returns
The number of samples stored in EEPROM.

Definition at line 1027 of file storage.c.

Referenced by Storage_GetCount(), Storage_Seek(), and StoreSamplesInEeprom().

◆ GetFlashCount()

static int GetFlashCount ( void  )
static
Returns
The number of samples stored in FLASH.

Definition at line 1034 of file storage.c.

Referenced by MoveSamplesFromEepromToFlash(), and Storage_GetCount().

◆ StoreSamplesInEeprom()

static int StoreSamplesInEeprom ( const STORAGE_TYPE pSamples,
int  n 
)
static

Stores n samples in EEPROM. If necessary, the oldest data is moved to FLASH.

Parameters
pSamplesMay not be NULL. Pointer to the start of the array where to copy the samples from.
nThe size of the array
Returns
The number of samples that were stored. A value less than n indicates insufficient storage capacity.

Definition at line 1110 of file storage.c.

Referenced by Storage_Write().

◆ MoveSamplesFromEepromToFlash()

static bool MoveSamplesFromEepromToFlash ( void  )
static

Clear the assigned EEPROM region, by moving all data to assigned FLASH region. Before moving the data, the application is given the opportunity to compress the data. The (compressed) data block is then appended to the existing data in FLASH. Steps:

  • Prepare STORAGE_WORKAREA for FLASH write
  • Copy data from EEPROM to STORAGE_WORKAREA
  • Call STORAGE_COMPRESS_CB
  • Flash
  • Update pointers
    Returns
    true when the compression was successful and the data has been moved to FLASH, false when the compression callback function returned false or when the FLASH storage is full: nothing has been changed in FLASH or EEPROM in that case.
    Postcondition
    sInstance is fully updated when this function returns.
    Note
    Uses STORAGE_WORKAREA. Not in use anymore when this function returns.

Definition at line 1160 of file storage.c.

Referenced by StoreSamplesInEeprom().

◆ ReadAndCacheSamplesFromFlash()

static int ReadAndCacheSamplesFromFlash ( int  readCursor)
static

Reads a block of compressed data containing STORAGE_BLOCK_SIZE_IN_SAMPLES samples, decompresses the data and stores the result in the workspace given by the application.

Parameters
readCursorThe offset in bytes relative to FLASH_FIRST_BYTE_ADDRESS to the header preceding the (compressed) data block that must be read.
Returns
  • When the samples are available in STORAGE_WORKAREA (either when the previous decompression was still valid and the decompression callback was not called; or when decompression was successful as indicated by the returnvalue of the called decompression callback): the size of the (compressed) data block including the header. This is equal to the number of bytes to advance the read cursor to the header of the next (compressed) data block.
  • 0 when the FLASH contents were invalid, or when the decompression callback function returned false: nothing has been changed in that case.
Postcondition
Only Storage_Instance_t.cachedBlockOffset is fully updated when this function returns.
Note
Uses STORAGE_WORKAREA. Only the first STORAGE_UNCOMPRESSED_BLOCK_SIZE_IN_BYTES bytes are in use when this function returns.

Definition at line 1300 of file storage.c.

Referenced by Storage_Read().

◆ ValidateRecoverInfo()

static bool ValidateRecoverInfo ( void  )
static

Checks whether spRecoverInfo contains possible correct information.

Returns
true when the information contained in the structure spRecoverInfo points to can be used.

Definition at line 1336 of file storage.c.

Referenced by Storage_Init().

◆ ValidateMarker()

static bool ValidateMarker ( const Marker_t pMarker,
int  expectedFlashByteCursor 
)
static

Checks whether the given structure contains possible correct information.

Parameters
pMarker: May not be NULL. Points to the structure to check.
expectedFlashByteCursor: Ignored when negative. When zero or positive, provides an additional constraint on the marker structure.
Returns
true when the information contained in the structure can be used.

Definition at line 1351 of file storage.c.

Referenced by FindMarker(), and Storage_Init().

◆ ValidateHint()

static bool ValidateHint ( const Hint_t pHint)
static

Checks whether the given structure contains possible correct information.

Parameters
pHint: May not be NULL. Points to the structure to check.
Returns
true when the information contained in the structure can be used.

Definition at line 1369 of file storage.c.

Referenced by Storage_DeInit(), and Storage_Init().

◆ WriteHint()

static void WriteHint ( void  )
static

Uses the information of sInstance to create a Hint_t structure, and writes it to EEPROM.

Definition at line 1397 of file storage.c.

Referenced by MoveSamplesFromEepromToFlash(), Storage_DeInit(), and Storage_Init().

◆ WriteMarker()

static void WriteMarker ( void  )
static

Uses the information of sInstance to create a Marker_t structure, and writes it to EEPROM.

Definition at line 1417 of file storage.c.

Referenced by Storage_DeInit(), and Storage_Init().

Variable Documentation

◆ _etext

const int _etext

Generated by the linker, used to calculate STORAGE_FLASH_FIRST_PAGE

Referenced by Storage_Init().

◆ _data

const int _data

Generated by the linker, used to calculate STORAGE_FLASH_FIRST_PAGE

Referenced by Storage_Init().

◆ _edata

const int _edata

Generated by the linker, used to calculate STORAGE_FLASH_FIRST_PAGE

Referenced by Storage_Init().

◆ sStorageFlashFirstPage

int sStorageFlashFirstPage
static

Definition at line 394 of file storage.c.

Referenced by Storage_Init().

◆ checkSizeOfSampleType

int checkSizeOfSampleType[(sizeof(STORAGE_TYPE) *8< STORAGE_BITSIZE) ? -1 :1]

If this construct doesn't compile, STORAGE_TYPE can not hold STORAGE_BITSIZE bits.

Definition at line 611 of file storage.c.

◆ checkSizeOfHint

int checkSizeOfHint[(SIZE_OF_HINT==sizeof(Hint_t)) ? 1 :-1]

If this construct doesn't compile, the define SIZE_OF_MARKER is no longer equal to sizeof(Hint_t)

Definition at line 614 of file storage.c.

◆ checkSizeOfMarker

int checkSizeOfMarker[(SIZE_OF_MARKER==sizeof(Marker_t)) ? 1 :-1]

If this construct doesn't compile, the define SIZE_OF_MARKER is no longer equal to sizeof(Marker_t)

Definition at line 617 of file storage.c.

◆ checkSizeOfRecoverInfo

int checkSizeOfRecoverInfo[sizeof(RecoverInfo_t)==4 ? 1 :-1]

If this construct doesn't compile, adjust FIRST_BITS_OF_CACHE_SIZE. RecoverInfo_t must be exactly 32 bits in size

Definition at line 620 of file storage.c.

◆ sInstance

Storage_Instance_t sInstance
static

The module's instance information, where recovered information is copied to, and where all updates are stored. In Storage_DeInit, this information is copied to more permanent storage.

Definition at line 637 of file storage.c.

Referenced by GetEepromCount(), GetFlashCount(), MoveSamplesFromEepromToFlash(), ReadAndCacheSamplesFromFlash(), ResetInstance(), Storage_DeInit(), Storage_Init(), Storage_Read(), Storage_Reset(), Storage_Seek(), Storage_Write(), StoreSamplesInEeprom(), WriteHint(), and WriteMarker().

◆ sCache

uint8_t sCache[4 *(5 - STORAGE_FIRST_ALON_REGISTER)]
static

To reduce the call count to Chip_PMU_GetRetainedData and Chip_PMU_SetRetainedData, data is copied to SRAM in Storage_Init once, and copied back to GPREG in Storage_DeInit once.

Definition at line 644 of file storage.c.

Referenced by CacheSample(), GetCachedSample(), Storage_DeInit(), and Storage_Init().

◆ spRecoverInfo

RecoverInfo_t* spRecoverInfo
static

Initialized in Storage_Init to points to the recovery structure in sCache.

Definition at line 650 of file storage.c.

Referenced by CacheSample(), GetCachedSample(), Storage_DeInit(), Storage_GetCount(), Storage_Init(), Storage_Reset(), Storage_Write(), and ValidateRecoverInfo().

◆ sEepromBitCursorChanged

bool sEepromBitCursorChanged = false
static

Set to true after writing a new sample in EEPROM, used in Storage_DeInit to know when to write the marker. Not set when resetting the module; as there is no need for a marker in that case.

Definition at line 658 of file storage.c.

Referenced by Storage_DeInit(), and StoreSamplesInEeprom().

◆ sStorage_Workarea

uint8_t sStorage_Workarea[STORAGE_WORKAREA_SIZE]

Definition at line 662 of file storage.c.

◆ STORAGE_WORKAREA