TN200302B - Internet Telemetry

Vaughan Wesson, February 2003.

 

Introduction

This technical note covers the general philosophy of using the Internet to transfer data, the various components in the system and then describes in detail how this is implemented with the Kelunji D Series recorder.

A useful glossary of Internet terms is available as TN200302D.

Internet telemetry is the term used by the SRC to describe the use of the Internet (or more precisely IP based communications) as a significant link in the chain between a recorder and the analysis centre. There are a number of advantages of this system including:

Components of the System

The components of the system can generally be described as the chain:

Kelunji <-> ISP <-> Internet <-> ISP <-> Analysis Centre

That is, the Kelunji connects to an Internet Service Provider (ISP) which is connected to the Internet. Similarly, the analysis centre (or data management centre or whatever) is also connected via an ISP to the Internet. The IP based protocols (see glossary) are used to transfer (route) the information from the Kelunji, through its ISP, through the Internet, through the other ISP to the analysis centre and vice versa.

The Internet

The Internet is often drawn as a fussy cloud or some other vague shape as it is difficult to describe it as having any physical form. It is a collection of cables and satellite links providing a series of communication channels between devices. It is the Internet Protocol (IP) that is the common factor allowing all the various devices to communicate with each other.

Internet Service Providers

The ISP's provide the ability for normal users to connect to The Internet. They will typically have more than one communication channel to The Internet backbone to provide a secure service and allow users to connect to them using various means. The most common means are:

For example, ES&S has an ADSL connection to its ISP, while at home I have a cable connection to mine and many of our seismographs use a dial up modem connection. The main differences between the systems are speed, reliability and cost.

A useful resource for obtaining details about local ISP's in Australia can be found here. With this site you can search for an ISP using either a phone number or geographically.

If just sending triggered data files, a cheap Internet account may be used. Either one based on connection time (around $1 per hour) or a fixed monthly account with a limited number of hours can be used. However, for continuous data a more expensive account allowing for continuous connection is required. Finally, a fixed IP address for the site will usually cost more again.

Analysis Centre

The analysis centre is where all the data ends up. For the Kelunji Internet telemetry services, all that is required in (or associated with) the analysis centre is an FTP server. This is what the Kelunji will send the data files to. The Kelunji has settings for a user name and password for connecting to the FTP server and a directory may also be specified for where the files are to be stored. As of version V5.11A, the Kelunji supports both active and passive mode FTP file transfers, so should work with all FTP servers.

To make best use of the data, the SRC's eqLogger and eqWatch programs are designed to process continuous and triggered files respectively as produced by the Kelunji.

Kelunji Data Communications via the Internet

The Kelunji contains software to perform IP based communications and can therefore connect directly to an ISP or a computer. At present, Guria (V5.10A through to V5.11A) provides the following Internet based services:

In the near future, we will be adding email support. This will provide some or all of the following additional services:

The listed services are available through either an Ethernet connection or a serial connection using PPP. The Ethernet interface is faster but uses more power and may not be available in some situations. The PPP interface will probably be the most common in the field.

Network Interfaces

The Kelunji Internet connectivity is provided through what it calls its "Network interfaces". These are set up in the "Settings:Serial:Network interfaces" menu. For versions up to at least V5.11A only one interface is supported, but the code is written to support up to four separate interfaces - for example, one Ethernet and three PPP. Each interface may be operated in one of three modes:

Mode 0 - Permanent
In this mode, the Kelunji will try and keep the interface open at all times. For example if a phone call through a modem is dropped for some reason, the software will recognise this after some time and attempt to re-establish the connection.
Mode 1 - On Demand
In this mode, the Kelunji will normally have the modem set waiting to answer a phone call and establish a PPP connection. However, when it is required to send a file (e.g. after triggering), it will initiate a connection to the ISP as specified. After the file has been transferred, the connection will be closed and the Kelunji will revert to waiting for a phone call.
Mode 2 - Send Only
In this mode, the connection is usually closed. When it is required to send a file, the connection is opened, the file sent and then the connection is closed again.

Obviously, the mode to use will depend on what services are being used. Mode 0 (permanent) is the usual mode if continuous data files are being sent or either the FTP or Web server is required. Mode 2 (send only) will yield the lowest cost and power consumption if using a modem powered from the Kelunji.

Each interface has its own IP address. This may be obtained as part of the PPP negotiation process. It also has its own network mask, gateway address and up to three DNS server addresses. The PPP negotiation will also attempt to obtain a DNS server address from the ISP. The DNS servers are queried in turn to convert a domain name to an IP address if required.

PPP Interfaces

The Kelunji contains up to four serial ports that may be used for PPP connections. These are the Console ("Serial") port and the three Digitiser ports. Where possible, it is best to leave the Console port for a user terminal and use one of the Digitiser ports for the modem to connect to the ISP.

For PPP interfaces additional settings are available.

The limits on the phone calls are required to cope with situations where the Kelunji cannot establish a connection to the ISP for some reason. The first limit is the number of calls allowed in one 24 hour period from 0000 UT one day to 0000 UT the next day. Each time the Kelunji attempts to dial the ISP is considered one phone call (note that it does not have to actually connect for it to be counted). The second parameter forces a minimum time between phone calls. This is designed to cope with periods of congestion on the GSM network for example. The time is measured from when the previous call was attempted.

Ethernet Interface

A single 10 Mbit/s Ethernet interface is provided on the Kelunji recorder. This can be used for either twisted pair or coaxial cables. With the Ethernet enabled, the Kelunji power consumption increases considerably. This is from a combination of the power drawn by the Ethernet transceiver itself and the fact that the processor is required to run at high speed all the time rather than sleeping when no tasks are to be performed.

When using the Ethernet interface, the network mask and gateway address will usually be required.

An Ethernet port has a unique address assigned to it by the manufacturer, known as its MAC address. This is a 48 bit number usually written as six 8 bit numbers each written in hex and separated by either a hyphen or a colon. For example 08:00:69:02:1D:FC or 08-00-69-02-1D-FC. Each manufacturer obtains an allocation of address from the Institute of Electrical and Electronics Engineers, Inc (IEEE) usually by obtaining an "Organisationally Unique Identifier" (OUI) which is a three byte value that may be used as the first three bytes of an Ethernet address. Environmental Systems and Services has the OUI of 00-0B-22. For the Kelunji D series, its Ethernet MAC address is obtained by combing our OUI with the serial number of the DK board. Therefore the MAC address of DK board number 77 would be 00-0B-22-00-00-4D.

File Transfer Steps

This section lists the steps the Kelunji takes in sending a file via FTP. It includes the messages that may appear if the diagnostic debug messages are enabled.

There is a task in the recorders software that is dedicated to sending files via FTP. This task may be "triggered" by any of the following events:

Any files to be sent via FTP are written to either the FTP_CONT or FTP_TRIG directories. Continuous data files are stored in the FTP_CONT directory, all others are in the FTP_TRIG directory. Note that when using multiple drives, these directories will be present on ALL drives.

However the task is triggered, the same steps are followed. These are:

Check and Send Files From a Directory

Step Possible Debug Messages Action taken on Failure
For each drive in turn, count the number of files in the directory. A maximum of 50 files will be processed at a time. These are not necessarily the earliest 50, they are just processed in the order they appear in the directory system    
If some files are found on this drive, send them (see below) "Only sent xx of the yy files successfully" if not all files were sent Ignored

Send a Group of Files

Step Possible Debug Messages

Action Taken on Failure

Sort the files by file name. This also means that they are sorted chronologically.    
Check whether the interface is being used by a dial in call. "Interface already in use" Transfer aborted
Check whether we are allowed to make a phone call. If the interface is not using a modem, this will always be true. Similarly, if the interface is already open we are OK. Otherwise, the time since the last call is checked and then the number of calls this day. "Not enough time elapsed for i/f #x" or "Too many calls this day for i/f #x" Transfer aborted
Attempt to open the Interface (see below) If the open succeeded, "Opened interface #x through Ethernet port" or "Opened interface #x to phonenumber, username, password". If the open failed, "Could not open interface #x to phonenumber, username, password" Transfer aborted
For PPP links, check that the connection is still "up" using an LCP echo request   Transfer aborted
Lookup the IP address specified for the FTP server "Resolved address domain.name to nnn.nn.nn.nn" or "Could not resolve address domain.name" Transfer aborted
Attempt to connect to the FTP server "Connect failed, status=nn". The common status values are -3 for a bad password, -4 for a bad user name or -1 for a general connection problem. If the interface is in Mode 0 (Permanent) and is a PPP interface, then the connection is closed as we may well have lost our connection. File transfers are abandoned.
Set the FTP file transfer mode to image (binary) "Problem setting type to image, status=n" Quit the FTP server and abort transfers
If a directory is specified, ask the FTP server to change to the indicated directory "Problem setting the directory to xxxx, status=n". A status value of -1 indicates that the directory does not exist or the user does not have permission Quit the FTP server and abort transfers
Send each file in turn (see below). Delete the file once it has been successfully sent. "Sending filename" Ignored. Move on to next file
Close the FTP session    
If the interface is not to be left open, then we now close it (see below)    

Open an Interface

Step Possible Debug Messages

Action Taken on Failure

If it is already open then we are done    
If it was opening waiting for a phone call then we must close it first    
If the interface is already in the process of opening (having being triggered already) then we abandon trying to open it again    
Open an Ethernet interface or a PPP interface as required (see below)    
Wait until the interface is open. For a PPP interface, this may take some time as this includes the modem dialling and training with its remote end, logging in to the ISP and the PPP handshaking.   Hang up the modem and indicate failure
If the interface opened OK, we enter any DNS servers associated with that interface to the active DNS server table and we are done    

Open an Ethernet Interface

Step Possible Debug Messages

Action Taken on Failure

Enable the Ethernet transceiver and set up the Ethernet port   Give up
Set the IP address and network mask for the port    
Notify the Web server of the new IP address    
Add the gateway address to the routing table    

Open a PPP Interface

Step Possible Debug Messages

Action Taken on Failure

Check that we have access to the indicated serial port   Give up
Check that PPP links have not been disabled (using the Disarm button)   Give up
Check whether the interface is already opening   Give up
Spawn a separate task to perform the connection   Give up
Open the serial port at the desired sample rate   Return failure indication
Set the IP address and network mask for the port   Return failure indication
Set up the PPP negotiation constraints    
Switch on the modem power if required and wait the indicated delay    
Dial the modem and attempt to connect to the ISP (see below) Many. See table below Hangup the modem, switch off modem power and return failure indication
Perform the PPP negotiation A number are possible from the RTIP software Hangup the modem, switch off modem power and return failure indication
Add the remote address as a default gateway in the routing table   Hangup the modem, switch off modem power and return failure indication
Notify the Web server of the new IP address    

Dial Modem and Connect to ISP

Step Possible Debug Messages

Action Taken on Failure

Update details about number of calls and time of last call    
Send init string to modem

The various modem exchanges and "Dialling phonenumber failed"

Hang up and return failure
Send dial string to modem The various modem exchanges Hang up and return failure
On connection, note the call start time    

Send a File

Step Possible Debug Messages

Action Taken on Failure

Open the file "Problem opening the file to send, status=n". Return failure
Transfer the file giving it a temporary name on the destination machine FTP commands and answers. "Problem sending the file, status=n". Return failure
Rename the file on the destination machine FTP commands and answers. "Rename from failed. status=n", "Rename to failed. Status=n". Return failure

Close the Interface

Step Possible Debug Messages

Action Taken on Failure

If it is already closed then we are done    
If it is an Ethernet interface, disable the transceiver and close down the Ethernet controller    
If it is a PPP interface, signal any open connections to close then wait for the PPP link to shut down. Hang up the modem, close the serial port and switch off power to the modem    
Remove any DNS servers from the active list that were for this interface    


Copyright © 2003, Seismology Research Centre
Last modified: 2004-08-31