Click to email me

Have I got time for another project? No, not really – but what the heck!  From time-to-time in my work I have a need to analyze public water supply pressures. For this I need periodic measurements of pressure over several weeks. The data is currently recorded using a circular chart recorder, but the thickness of the pen trace equates to 10’s of minutes on the polar axis and 10’s of kPa on the radial axis. See Figure 1.  Further the electromechanical recorder’s damping (or should that be overshoot) make transients difficult to interpret. Reading the circular charts is a pain at the best of times and interpreting the results is an art rather than science.

Figure 1.  Polar Pressure Plot

I have also been thinking about a high speed data logger for recording ultrasonic sound underwater to assist with the development of the Buddy Locator.

The pressure recorder is a typical data logger application.  The logger needs to be battery powered, accept up to eight input channels, incorporate a real time clock, provide at least 1% reading resolution, be capable of calibration, provide storage for at least two weeks of data taken at one minute intervals, and allow straight forward analysis using software such as MS Excel.

There are some very low cost basic SD card loggers available.  However the commercial units suitable for the pressure reading application are expensive and I have not been able to find something that is suitable for ultrasonic logging.  I decided to make one as a learning exercise and as an initial foray into high speed data acquisition.

A standard SD card formatted with FAT16 was selected as the data storage media because they are cheap, robust, small, and readily interface to a PC.

The next job was selecting a microprocessor. I settled on an Atmel ATMega 163L which is regrettably now obsolete.  I chose this processor because I am familiar with the instruction set, I had one available (along with an Atmel SDK500 development board); it has sufficient SRAM for SD card sector reads, a 10 bit eight channel Analogue to Digital Converter (ADC), an SPI interface for an SD card interface and sufficient flash memory to allow development of a FAT16 file system.  The chip can be readily replaced, with minimal software changes, to more recent versions of the ATMega range such as the ATMega 16.

The hardware was designed with an LCD display, Real time Clock (RTC), four push buttons, 8 channel ADC, an LED and a serial port interface SD card interface through the SPI bus.

The board was developed using Eagle PCB software and included on-board regulated power supply, polarity protection, and filtering. The board was made with an upper ground-plane to reduce noise on the ADC channels and eliminate jumpers.  See Figure 2.

The board was constructed and populated. Software was developed in stages to test the hardware: first the LED, then the push buttons, then the LCD, followed by the RTC, then the serial port followed by the SD card interface, and finally the FAT16 file system.  While testing the hardware I wrote the primary subroutines in assembly language using AVR Studio.

Figure 2.  Data Logger

For development of the SPI to SD Card interface I give full credit to ‘the captain’. The FAT16 routines were developed using information from Advanced Assembly Language by S. Holzner and P. Norton and Undocumented DOS by Schulman, Michaels, Kyle Paterson, Maxy and Brown.  My initial SD card hardware test software allowed me to dump the SD Card sector-by-sector to my PC.  There was a time when operating systems allowed utilities such as Debug to access storage structures directly. Alas Windows XP doesn’t allow this any more.

With all of the hardware working and tested I’ve started on the FAT16 file interface.  A significant problem is the capacity of MS Excel – 2^16 (65,536) rows of data * 2^8 (256) columns. With one minute interval readings in each row this gives a maximum of 45 days of data.  It is intended to save the file as an ASCII text file using tab separation between readings. Each set of readings is to be preceded by date and time rows and terminated with carriage return and line feed characters.  The maximum file size is therefore ~ 4 Megabytes corresponding with 7,680 * 512 byte sectors, or 480 * 16 sector clusters.

The software is developing somewhat slower than expected.  While the hardware interface routines were relatively easy to implement, the FAT interface has been somewhat more challenging due to the large number of 16 and 32 bit variables that are required to keep track of the FAT.  As at today the code is over 2000 instructions and I figure I have another 1000 to go. My unfamiliarity with the development environment, AVR Studio, and C have not helped. However I am making steady progress each evening and it will not be too long before I have completed the project in assembly language.

It’s 21 October already - where is the year going? Life, the development of my Buddy Locator, my A20 camera housing, a slave underwater flash, and messing around with a robot have taken prescidence over this project for too long.  I sat down today and started rethinking the file system design. I am faced with three competing requirements, data security, write speed, and MS Excel compatibility.  At this time I have a hardware platform that cannot be readily modified to allow faster data transfer.  (If I’d thought about this more at the outset the design would have included a separate ADC, a paged ram cache with its own MCU, and a storage device using the full width SD Card bus.)

The current design approach has been based upon a full FAT16 implementation with file updates on each 512 byte block store.  The limited RAM resource and the speed of the MCU are limiting factors.

Another option open to me is to ignore the FAT format. This way I can write 512 byte blocks of data directly to the SD card,  read it back, block by block, convert it to an appropriate form, and send it via the serial port to a PC.  This would certainly be the fastest option for the current hardware and ensure data security for all but the last 512 byte block. It is also the easiest solution to implement in software.

I have had another thought.  Why not initialize a full 4 MB Excel file using the FAT16 format? Then I can write ASCII converted data full-tilt to the SD card without the file management read/write overheads.  If the file consists of sequential sectors then the write process should be very straight forward. This will preserve the data security to the last 512 byte block while maintaining MS Excel compatibility, with optimized write speed for the current hardware.

24 Oct and I have started porting my assembler routines to C using WINavr. At best I am a C novice and this experience has been painful. Missing “}” and “_, and a lack of understanding of the dreaded Makefile have resulted in days of compiler errors and increased hair loss (let’s face it, I had little enough to start with).  I have been using a collection of references: “C Programing for Microcontrollers” by Joe Pardue; “Embeded C Programming and the Atmel AVR”, by Barnett, Cox and O’Cull; and several books with the generic title C/C++ in ?? days.  Alas the literature has not kept pace with the Integrated Development Environment so changes in the header files have been frequent causes of grief.   That said I have used as many applicable code segments as I can find from the literature and the net in the hope of producing something that works this year.

I have managed to get the basic initialization and hardware functions implemented, compiled error free, and simulated in WINavr but I haven’t transferred these to the hardware platform yet.  The code has blown out from 2,000 odd bytes in assembler to over 5,000 bytes in C. I’m rather hoping that the ‘include’ library overhead will not increase appreciably from here.

More to follow...

[Turtles are Here] [Construction] [Drawings] [Buddy Locator] [Links] [Photo Galleries] [A20 Housing] [Data Logger] [Contact Me] [Galapagaos] [Talk Underwater] [Making PCBs]