ESP32 WiFi

WiFi subsystem

WiFi Theory

When working with a WiFi oriented device, it is important that we have at least some understanding of the concepts related to WiFi. At a high level, WiFi is the ability to participate in TCP/IP connections over a wireless communication link. WiFi is specifically the set of protocols described in the IEEE 802.11 Wireless LAN architecture.

Within this story, a device called a Wireless Access Point (access point or AP) acts as the hub of all communications. Typically it is connected to (or acts as) as TCP/IP router to the rest of the TCP/IP network. For example, in your home, you are likely to have a WiFi access point connected to your modem (cable or DSL). WiFi connections are then formed to the access point (through devices called stations) and TCP/IP traffic flows through the access point to the Internet.

The devices that connect to the access points are called "stations":

An ESP32 device can play the role of an Access Point, a Station or both at the same time.

Very commonly, the access point also has a network connection to the Internet and acts as a bridge between the wireless network and the broader TCP/IP network that is the Internet.

A collection of stations that wish to communicate with each other is termed a Basic Service Set (BSS). The common configuration is what is known as an Infrastructure BSS. In this mode, all communications inbound and outbound from an individual station are routed through the access point.

A station must associate itself with an access point in order to participate in the story. A station may only be associated with a single access point at any one time. Each participant in the network has a unique identifier called the MAC address. This is a 48bit value.

When we have multiple access points within wireless range, the station needs to know with which one to connect. Each access point has a network identifier called the BSSID (or more commonly just SSID). SSID is service set identifier. It is a 32 character value that represents the target of packets of information sent over the network.

See also: • Wikipedia – Wireless access point • Wikipedia – IEEE 802.11 • Wikipedia – WiFi Protected Access • Wikipedia – IEEE 802.11i-2004

Initializing the WiFi environment

WiFi is only a part of the capabilities of an ESP32. As such, there may be some times when you actually don't want to use the WiFi subsystem. To accommodate those patterns, the initialization of the WiFi subsystem is expected to be performed by you when you write your applications. This is done by calling the esp_wifi_init() method.

The recommended way of doing this is as follows: wifi_init_config_t config = WIFI_INIT_CONFIG_DEFAULT(); esp_wifi_init(&config); See also: • esp_wifi_init

Setting the operation mode

The ESP32 can either be a station in the network, an access point for other devices or both. Remember, when an ESP32 is being a station, it can connect to a remote access point (your WiFi hub) while when being an access point, other WiFi stations can connect to the ESP32 (think of the ESP32 as becoming a WiFi hub). This is a fundamental consideration and we will want to choose how the device behaves early on in our application design. Once we have chosen what we want, we set a global mode property which indicates which of the operational modes our device will perform (station, access point or station AND access point).

This choice is set with a call to esp_wifi_set_mode(). The parameter is an instance of wifi_mode_t which can have a value of WIFI_MODE_NULL, WIFI_MODE_STA, WIFI_MODE_AP or WIFI_MODE_APSTA. We can call esp_wifi_get_mode() to retrieve our current mode state.

Scanning for access points

If the ESP32 is going to be performing the role of a station we will need to connect to an access point. We can request a list of the available access points against which we can attempt to connect. We do this using the esp_wifi_scan_start() function.

The results of a WiFi scan are stored internally in ESP32 dynamically allocated storage.

The data is returned to us when we call esp_wifi_scan_get_ap_records() which also releases the internally allocated storage. As such, this should be considered a destructive read.

A scan record is contained in an instance of a wifi_ap_record_t structure that contains: uint8_t bssid[6] uint8_t ssid[32] uint8_t primary wifi_second_chan_t second int8_t rssi wifi_auth_mode_t authmode

The wifi_auth_mode_t is one of: • WIFI_AUTH_OPEN – No security. • WIFI_AUTH_WEP – WEP security. • WIFI_AUTH_WPA_PSK – WPA security. • WIFI_AUTH_WPA2_PSK – WPA2 security. • WIFI_AUTH_WPA_WPA2_PSK – WPA or WPA2 security.

After issuing the request to start performing a scan, we will be informed that the scan completed when a SYSTEM_EVENT_SCAN_DONE event is published. The event data contains the number of access points found but that can also be retrieved with a call to esp_wifi_scan_get_ap_num().

Should we wish to cancel the scanning before it completes on its own, we can call esp_wifi_scan_stop().

Here is a complete sample application illustrating performing a WiFi scan. Much of the work is performed in the event handler. When we detect a scan completion event, we retrieve the located access points and log their details.

1. include "esp_wifi.h"
2. include "esp_system.h"
3. include "esp_event.h"
4. include "esp_event_loop.h"
5. include "nvs_flash.h"

esp_err_t event_handler(void *ctx, system_event_t *event) { if (event->event_id == SYSTEM_EVENT_SCAN_DONE) { printf("Number of access points found: %d\n", event->event_info.scan_done.number); uint16_t apCount = event->event_info.scan_done.number; if (apCount == 0) { return ESP_OK; } wifi_ap_record_t *list = (wifi_ap_record_t *)malloc(sizeof(wifi_ap_record_t) * apCount); ESP_ERROR_CHECK(esp_wifi_scan_get_ap_records(&apCount, list)); int i; for (i=0; i<apCount; i++) { char *authmode; switch(list[i].authmode) { case WIFI_AUTH_OPEN: authmode = "WIFI_AUTH_OPEN"; break; case WIFI_AUTH_WEP: authmode = "WIFI_AUTH_WEP"; break; case WIFI_AUTH_WPA_PSK: authmode = "WIFI_AUTH_WPA_PSK"; break; case WIFI_AUTH_WPA2_PSK: authmode = "WIFI_AUTH_WPA2_PSK"; break; case WIFI_AUTH_WPA_WPA2_PSK: authmode = "WIFI_AUTH_WPA_WPA2_PSK"; break; default: authmode = "Unknown"; break; } printf("ssid=%s, rssi=%d, authmode=%s\n", list[i].ssid, list[i].rssi, authmode); } free(list); } return ESP_OK; } int app_main(void) { nvs_flash_init(); tcpip_adapter_init(); ESP_ERROR_CHECK(esp_event_loop_init(event_handler, NULL)); wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT(); ESP_ERROR_CHECK(esp_wifi_init(&cfg)); ESP_ERROR_CHECK(esp_wifi_set_storage(WIFI_STORAGE_RAM)); ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA)); ESP_ERROR_CHECK(esp_wifi_start()); // Let us test a WiFi scan ... wifi_scan_config_t scanConf = { .ssid = NULL, .bssid = NULL, .channel = 0, .show_hidden = 1 }; ESP_ERROR_CHECK(esp_wifi_scan_start(&scanConf, 0)); return 0; } Using the Arduino libraries we can also make network scans. Here is an example: int8_t count = WiFi.scanNetworks(); printf("Found %d networks\n", count); for (uint8_t i=0; i<count; i++) { String ssid; uint8_t encryptionType; int32_t RSSI; uint8_t *BSSID; int32_t channel; WiFi.getNetworkInfo(i, ssid, encryptionType, RSSI, BSSID, channel); printf("ssid=%s\n", ssid.c_str()); } See also: • Handling WiFi events • esp_wifi_scan_start • esp_wifi_scan_stop • esp_wifi_scan_get_ap_records • esp_wifi_scan_get_ap_num

Handling WiFi events

During the course of operating as a WiFi device, certain events may occur that ESP32 needs to know about. These may be of importance or interest to the applications running within it. Since we don't know when, or even if, any events will happen, we can't have our application block waiting for them to occur. Instead what we should do is define a callback function that will be invoked should an event actually occur. The function called esp_event_loop_init() does just that. It registers a function that will be called when the ESP32 detects certain types of WiFi related events. The registered function is invoked and passed a rich data structure that includes the type of event and associated data corresponding to that event. The types of events that cause the callback to occur are: • We connected to an access point • We disconnected from an access point • The authorization mode changed • A station connected to us when we are in Access Point mode • A station disconnected from us when we are in Access Point mode • A SSID scan completes

When the ESP32 WiFi environment operates, it publishes "events" when something at the WiFi level occurs such as a new station connecting. We can register a callback function that is invoked when an event is published. The signature of the callback function is: esp_err_t eventHandler(void *ctx, system_event_t *event) { // Handle event here ... return ESP_OK; } Typically, we need to also include the following: • #include <esp_event.h> • #include <esp_event_loop.h> • #include <esp_wifi.h> • #include <esp_err.h>

To register the callback function, we invoke: esp_event_loop_init(eventHandler, NULL);

If we wish to subsequently change the event handler associated with our WiFi handling we can call: esp_event_loop_set_cb(eventHandler, NULL);

When the event handler is invoked, the event parameter is populated with details of the event. The data type of this parameter is a "system_event_t" which contains: system_event_id_t event_id system_event_info_t event_info We should include "esp_event.h" to gain access to these functions and definitions. Let us now look at the two properties passed to the event handler in the system_event_t data structure. These properties are "event_id" and "event_info". At a high level, the event_id describes what kind of event was detected while event_info contains the specific details of the event based on the type identified in event_id. • event_id – An enumeration type with the following potential values: ◦ SYSTEM_EVENT_WIFI_READY – ESP32 WiFi is ready. ◦ SYSTEM_EVENT_SCAN_DONE – Finished scanning for access points. The scan_done data field is valid to be accessed. ◦ SYSTEM_EVENT_STA_START – Started being a station. ◦ SYSTEM_EVENT_STA_STOP – Stopped being a station. ◦ SYSTEM_EVENT_STA_CONNECTED – Connected to an access point as a station. The connected data field is valid to be accessed. ◦ SYSTEM_EVENT_STA_DISCONNECTED – Disconnected from access point while being a station. The disconnected data field is valid to be accessed. ◦ SYSTEM_EVENT_STA_AUTHMODE_CHANGE – Authentication mode has changed. The auth_change data field is valid to be accessed. ◦ SYSTEM_EVENT_STA_GOT_IP – Got an assigned IP address from the access point that we connected to while being a station. The got_ip data field is valid to be accessed. ◦ SYSTEM_EVENT_AP_START – Started being an access point. ◦ SYSTEM_EVENT_AP_STOP – Stopped being an access point. ◦ SYSTEM_EVENT_AP_STACONNECTED – A station connected to us while we are being an access point. The sta_connected data field is valid to be accessed. ◦ SYSTEM_EVENT_AP_STADISCONNECTED – A station disconnected from us while we are being an access point. The sta_disconnected data field is valid to be accessed. ◦ SYSTEM_EVENT_AP_PROBEREQRECVED – Received a probe request while we are being an access point. The ap_probereqrecved data field is valid to be accessed. • event_info – This is a C language union of distinct data types that are keyed off the event_id. The different structures contained within are: Structure Field Event system_event_sta_connected_t connected SYSTEM_EVENT_STA_CONNECTED system_event_sta_disconnected_t disconnected SYSTEM_EVENT_STA_DISCONNECTED system_event_sta_scan_done_t scan_done SYSTEM_EVENT_SCAN_DONE system_event_sta_authmode_change_t auth_change SYSTEM_EVENT_STA_AUTHMODE_CHANGE system_event_sta_got_ip_t got_ip SYSTEM_EVENT_STA_GOT_IP system_event_ap_staconnected_t sta_connected SYSTEM_EVENT_AP_STACONNECTED system_event_ao_stadisconnected_t sta_disconnected SYSTEM_EVENT_AP_STADISCONNECTED system_event_ap_probe_req_rx_t ap_probereqrecved SYSTEM_EVENT_AP_PROBEREQRECVED These data structures contain information pertinent to the event type received.

system_event_sta_connected_t

This data type is associated with the SYSTEM_EVENT_STA_CONNECT event. uint8_t ssid[32] uint8_t ssid_len uint8_t bssid[6] uint8_t channel wifi_auth_mode_t authmode The ssid is the WiFi network name to which we connected. The ssid_len is the number of bytes in the ssid field that contain the name. The bssid is the MAC address of the access point. The channel is the wireless channel used for the connection. The authmode is the security authentication mode used during the connection.

system_event_sta_disconnected_t

This data type is associated with the SYSTEM_EVENT_STA_DISCONNECTED event. uint8_t ssid[32] uint8_t ssid_len uint8_t bssid[6] uint8_t reason The reason code is an indication of why we disconnected. Symbolics are defined for each of the numeric reason codes to allow us to write more elegant and comprehensible applications should we need to consider a reason code.: • WIFI_REASON_UNSPECIFIED – 1 • WIFI_REASON_AUTH_EXPIRE – 2 • WIFI_REASON_AUTH_LEAVE – 3 • WIFI_REASON_ASSOC_EXPIRE – 4 • WIFI_REASON_ASSOC_TOOMANY – 5 • WIFI_REASON_NOT_AUTHED – 6 • WIFI_REASON_NOT_ASSOCED – 7 • WIFI_REASON_ASSOC_LEAVE – 8 • WIFI_REASON_ASSOC_NOT_AUTHED – 9 • WIFI_REASON_DISASSOC_PWRCAP_BAD – 10 • WIFI_REASON_DISASSOC_SUPCHAN_BAD – 11 • WIFI_REASON_IE_INVALID – 13 • WIFI_REASON_MIC_FAILURE – 14 • WIFI_REASON_4WAY_HANDSHAKE_TIMEOUT – 15 • WIFI_REASON_GROUP_KEY_UPDATE_TIMEOUT – 16 • WIFI_REASON_IE_IN_4WAY_DIFFERS – 17 • WIFI_REASON_GROUP_CIPHER_INVALID – 18 • WIFI_REASON_PAIRWISE_CIPHER_INVALID – 19 • WIFI_REASON_AKMP_INVALID – 20 • WIFI_REASON_UNSUPP_RSN_IE_VERSION – 21 • WIFI_REASON_INVALID_RSN_IE_CAP – 22 • WIFI_REASON_802_1X_AUTH_FAILED – 23 • WIFI_REASON_CIPHER_SUITE_REJECTED – 24 • WIFI_REASON_BEACON_TIMEOUT – 200 • WIFI_REASON_NO_AP_FOUND – 201 • WIFI_REASON_AUTH_FAIL – 202 • WIFI_REASON_ASSOC_FAIL – 203 • WIFI_REASON_HANDSHAKE_TIMEOUT – 204

system_event_sta_scan_done_t

This data type is associated with the SYSTEM_EVENT_SCAN_DONE event. uint32_t status uint8_t number uint8_t scan_id See also: • Scanning for access points • esp_wifi_scan_get_ap_records

system_event_authmode_change_t

This data type is associated with the SYSTEM_EVENT_STA_AUTHMODE_CHANGE event. wifi_auth_mode_t old_mode wifi_auth_mode_t new_mode

system_event_sta_got_ip_t

This data type is associated with the SYSTEM_EVENT_STA_GOT_IP event. tcpip_adapter_ip_info_t ip_info The ip_info element is an instance of a tcpip_adapter_ip_info_t which contains three fields: • ip -The IP address. • netmask – The network mask. • gw – The gateway for communications. All three of these fields are of ip4_addr_t which is a 32bit representation of an IP address. During development, you might want to consider logging the IP address of the device. You can do this using: ESP_LOGD(tag, "Got an IP: " IPSTR, IP2STR(&event->event_info.got_ip.ip_info.ip));

system_event_ap_staconnected_t

This data type is associated with the SYSTEM_EVENT_AP_STACONNECTED event. uint8_t mac[6] uint8_t aid

system_event_ap_stadisconnected_t

This data type is associated with the SYSTEM_EVENT_AP_STADISCCONNECTED event. uint8_t mac[6] uint8_t aid

system_event_ap_probe_req_rx_t

This data type is associated with the SYSTEM_EVENT_AP_PROBREQRECVED event. int rssi uint8_t mac[6] If we enable the correct logging levels, we can see the events arrive and their content. For example: D (2168) event: SYSTEM_EVENT_STA_CONNECTED, ssid:RASPI3, ssid_len:6, bssid:00:00:13:80:3d:bd, channel:6, authmode:3 V (2168) event: enter default callback V (2174) event: exit default callback and D (9036) event: SYSTEM_EVENT_STA_GOTIP, ip:192.168.5.62, mask:255.255.255.0, gw:192.168.5.1 V (9036) event: enter default callback I (9037) event: ip: 192.168.5.62, mask: 255.255.255.0, gw: 192.168.5.1 V (9043) event: exit default callback

Station configuration

When we think of an ESP32 as a WiFi Station, we will realize that at any one time, it can only be connected to one access point. Putting it another way, there is no meaning in saying that the device is connected to two or more access points at the same time. The identity of the access point to which we wish to be associated is set within a data structure called wifi_sta_config_t. The wifi_sta_config_t contains: char ssid[32] char password[64] bool bssid_set uint8_t bssid[6] Contained within that structure are two very important fields called "ssid" and "password". The ssid field is the SSID of the access point to which we will connect. The password field is the clear text value of the password that will be used to authenticate our device to the target access point to allow connection. An example initialization for this structure might be: wifi_config_t staConfig = { .sta = { .ssid="<access point name>", .password="<password>", .bssid_set=false } }; Once we have populated an instance of this structure, we can instruct ESP32 about its content using: esp_wifi_set_config(WIFI_IF_STA, (wifi_config_t *)&staConfig); We should previously have called esp_wifi_set_mode(). with either: esp_wifi_set_mode(WIFI_MODE_STA) or esp_wifi_set_mode(WIFI_MODE_APSTA) See also: • esp_wifi_set_mode • esp_wifi_set_config

Starting up the WiFi environment

Since WiFi has states that it must go through, a question that may be asked is "When is WiFi ready to be used?". If we imagine that an ESP32 boots from cold, the chances are that we want to tell it to be either a station or an access point and then configure it with parameters such as which access point to connect to (if it is a station) or what its own access point identity should be (if it is going to be an access point). Given that these are a sequence of steps, we actually don't want the ESP32 to execute on these tasks until after we have performed all our setup. For example, if we boot an ESP32 and ask it to be an access point, if it started being an access point immediately then it may not yet know the details of the access point it should be or, worse, may transiently appear as the wrong access point. As such, there is final command that we must learn which is the instruction to the WiFi subsystem to start working. That command is esp_wifi_start(). Prior to calling that, all we are doing is setting up the environment. Only by calling esp_wifi_start() does the WiFi subsystem start doing any real work on our behalf. If our mode is that of an access point, calling this function will start us being an access point. If our mode is that of a station, now we are allowed to subsequently connect as a station. There is a corresponding command called esp_wifi_stop() which stops the WiFi subsystem. See also: • esp_wifi_start • esp_wifi_stop

Connecting to an access point

Once the ESP32 has been set up with the station configuration details which includes the SSID and password, we are ready to perform a connection to the target access point. The function esp_wifi_connect() will form the connection. Realize that this is not instantaneous and you should not assume that immediately following this command you are connected. Nothing in the ESP32 blocks and as such neither does the call to this function. Some time later, we will actually be connected. We will see two callback events fired. The first is SYSTEM_EVENT_STA_CONNECTED indicating that we have connected to the access point. The second event is SYSTEM_EVENT_STA_GOT_IP which indicates that we have been assigned an IP address by the DHCP server. Only at that point can we truly participate in communications. If we are using static IP addresses for our device, then we will only see the connected event. Should we disconnect from an access point, we will see a SYSTEM_EVENT_STA_DISCONNECTED event. To disconnect from a previously connected access point we issue the esp_wifi_disconnect() call. There is one further consideration associated with connecting to access points and that is the idea of automatic connection. There is a boolean flag that is stored in flash that indicates whether or not the ESP32 should attempt to automatically connect to the last used access point. If set to true, then after the device is started and without you having to code any API calls, it will attempt to connect to the last used access point. This is a convenience that I prefer to switch off. Usually, I want control in my device to determine when I connect. We can enable or disable the auto connect feature by making a call to esp_wifi_set_auto_connect(). Here is a complete sample illustrating all the steps needed to connect to an access point and be informed when we are ready to being work:

1. include "freertos/FreeRTOS.h"
2. include "esp_wifi.h"
3. include "esp_system.h"
4. include "esp_event.h"
5. include "esp_event_loop.h"
6. include "nvs_flash.h"
7. include "tcpip_adapter.h"

esp_err_t event_handler(void *ctx, system_event_t *event) { if (event->event_id == SYSTEM_EVENT_STA_GOT_IP) { printf("Our IP address is " IPSTR "\n", IP2STR(&event->event_info.got_ip.ip_info.ip)); printf("We have now connected to a station and can do things...\n") } return ESP_OK; } int app_main(void) { nvs_flash_init(); tcpip_adapter_init(); ESP_ERROR_CHECK( esp_event_loop_init(event_handler, NULL) ); wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT(); ESP_ERROR_CHECK(esp_wifi_init(&cfg) ); ESP_ERROR_CHECK( esp_wifi_set_storage(WIFI_STORAGE_RAM) ); ESP_ERROR_CHECK( esp_wifi_set_mode(WIFI_MODE_STA)); wifi_config_t sta_config = { .sta = { .ssid = "RASPI3", .password = "password", .bssid_set = 0 } }; ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &sta_config)); ESP_ERROR_CHECK(esp_wifi_start()); ESP_ERROR_CHECK(esp_wifi_connect()); return 0; } When we connect to an access point, our device is being a station. The connection to the access point doesn't automatically mean that we now have an IP address. We still have to request an allocated IP address from the DHCP server. This can take a few seconds. In some cases, we can get away with the device requesting a specific IP address. This results in a much faster connection time. If we do specify data, we also need to supply DNS information should we need to connect to DNS servers for name resolution. Here is a fragment of logic that allocates us a specific IP address:

1. include <lwip/sockets.h>

// The IP address that we want our device to have.

1. define DEVICE_IP "192.168.1.99"

// The Gateway address where we wish to send packets. // This will commonly be our access point.

1. define DEVICE_GW "192.168.1.1"

// The netmask specification.

1. define DEVICE_NETMASK "255.255.255.0"

// The identity of the access point to which we wish to connect.

1. define AP_TARGET_SSID "RASPI3"

// The password we need to supply to the access point for authorization.

1. define AP_TARGET_PASSWORD "password"

esp_err_t wifiEventHandler(void *ctx, system_event_t *event) { return ESP_OK; } // Code fragment here ... nvs_flash_init(); tcpip_adapter_init(); tcpip_adapter_dhcpc_stop(TCPIP_ADAPTER_IF_STA); // Don't run a DHCP client tcpip_adapter_ip_info_t ipInfo; inet_pton(AF_INET, DEVICE_IP, &ipInfo.ip); inet_pton(AF_INET, DEVICE_GW, &ipInfo.gw); inet_pton(AF_INET, DEVICE_NETMASK, &ipInfo.netmask); tcpip_adapter_set_ip_info(TCPIP_ADAPTER_IF_STA, &ipInfo); ESP_ERROR_CHECK(esp_event_loop_init(wifiEventHandler, NULL)); wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT(); ESP_ERROR_CHECK(esp_wifi_init(&cfg) ); ESP_ERROR_CHECK(esp_wifi_set_storage(WIFI_STORAGE_RAM)); ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA)); wifi_config_t sta_config = { .sta = { .ssid = AP_TARGET_SSID, .password = AP_TARGET_PASSWORD, .bssid_set = 0 } }; ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &sta_config)); ESP_ERROR_CHECK(esp_wifi_start()); ESP_ERROR_CHECK(esp_wifi_connect()); See also: • Handling WiFi events • esp_wifi_connect • esp_wifi_disconnect

Being an access point

So far we have only considered the ESP32 as a WiFi station to an existing access point but it also has the ability to be an access point to other WiFi devices (stations) including other ESP32s. In order to be an access point, we need to define the SSID that that allows other devices to distinguish our network. This SSID can be flagged as hidden if we don't wish it to be found in a scan. In addition, we will also have to supply the authentication mode that will be used when a station wishes to connects with us. This is used to allow authorized stations and disallow non-authorized ones. Only stations that know our password will be allowed to connect. If we are using authentication, then we will also have to choose a password that the connecting stations will have to know and supply to successfully connect. The first task in being an access point is to flag the ESP32 as such using the esp_wifi_set_mode() function and pass in the flag that requests we be either a dedicated access point or an access point and a station. This will be either: esp_wifi_set_mode(WIFI_MODE_AP); or esp_wifi_set_mode(WIFI_MODE_APSTA); Next we need to supply the configuration information. We do this by populating an instance of wifi_ap_config_t. The wifi_ap_config_t contains: • ssid – The WiFi ssid name upon which we will listen for connecting stations. • ssid_len – The length in bytes of the ssid if not NULL terminated. • password – The password used for station authentication. • channel – The channel we will use for networking. • authmode – How we wish stations to authenticate (if at all). The choices are ◦ open ◦ wep ◦ wpa ◦ wpa2 ◦ wpa_wpa2 • ssid_hidden – Should we broadcast our ssid. • max_connection – The number of concurrent stations. The default and maximum is 4. • beacon_interval – Unknown. 100. An example of initialization of this structure might be: wifi_config_t apConfig = { .ap = { .ssid="<access point name>", .ssid_len=0, .password="<password>", .channel=0, .authmode=WIFI_AUTH_OPEN, .ssid_hidden=0, .max_connection=4, .beacon_interval=100 } }; With the structure populated, we call esp_wifi_set_config() … for example: esp_wifi_set_config(WIFI_IF_AP, &apConfig); Finally, we call esp_wifi_start(). Here is a snippet of code that can be used to setup and ESP32 as an access point: When we become an access point, an ESP32 WiFi event is produced of type SYSTEM_EVENT_AP_START. Note that there is no payload data associated with this event. Once the ESP32 starts listening for station connects by being an access point, we are going to want to validate that this works. You can use any device or system to scan and connect. Personally, I use a Raspberry PI 3 for testing as it provides a nice Linux environment and has a WiFi adapter build in. You can also choose to plug in a separate WiFi dongle into one of the extra USB ports. One of the first tools we want to run is called "iwlist" which will perform a scan for us: $ sudo iwlist wlan1 scan In the results, we can look for our ESP32 … for example: Cell 02 - Address: 18:FE:34:6A:94:EF ESSID:"ESP32" Protocol:IEEE 802.11bgn Mode:Master Frequency:2.412 GHz (Channel 1) Encryption key:off Bit Rates:150 Mb/s Quality=100/100 Signal level=100/100 One of the other tools available on that environment is called "wpa_cli" which provides a wealth of options for testing WiFi. The recipe I use is to connect to an access point from the command line is: $ sudo wpa_cli add_network set_network <num> ssid "<SSID>" set_network <num> key_mgmt NONE enable_network <num> status You may have to run select_network <num> reconnect or reasociate to connect to the target and you can run disconnect to disconnect from the access point. ifname – show current interface interface <name> - select current interface To perform a scan run the command "scan". When complete, run "scan_results" to see the list. When a station connects, the ESP32 will raise the SYSTEM_EVENT_AP_STACONNECTED event. When a station disconnects, we will see the SYSTEM_EVENT_AP_DISCONNECTED event. See also: • man(8) – wpa_cl i When a remote station connects to the ESP32 as an access point, we will see a debug message written to UART1 that may look similar to: station: f0:25:b7:ff:12:c5 join, AID = 1 This contains the MAC address of the new station joining the network. When the station disconnects, we will see a corresponding debug log message that may be: station: f0:25:b7:ff:12:c5 leave, AID = 1 From within the ESP32, we can determine how many stations are currently connected with a call to wifi_softap_get_station_num(). If we wish to find the details of those stations, we can call wifi_softap_get_station_info() which will return a linked list of wifi_sta_list_t. We have to explicitly release the storage allocated by this call with an invocation of wifi_softap_free_station_info(). Here is an example of a snippet of code that lists the details of the connected stations: uint8 stationCount = wifi_softap_get_station_num(); os_printf("stationCount = %d\n", stationCount); wifi_sta_list_t *stationInfo = wifi_softap_get_station_info(); if (stationInfo != NULL) { while (stationInfo != NULL) { os_printf("Station IP: %d.%d.%d.%d\n", IP2STR(&(stationInfo->ip))); stationInfo = STAILQ_NEXT(stationInfo, next); } wifi_softap_free_station_info(); } When an ESP32 acts as an access point, this allows other devices to connect to it and form a WiFi connection. However, it appears that two devices connected to the same ESP32 acting as an access point can not directly communicate between each other. For example, imagine two devices connecting to an ESP32 as an access point. They may be allocated the IP addresses 192.168.4.2 and 192.168.4.3. We might imagine that 192.168.4.2 could ping 192.168.4.3 and visa versa but that is not allowed. It appears that they only direct network connection permitted is between the newly connected stations and the access point (the ESP32) itself. This seems to limit the applicability of the ESP32 as an access point. The primary intent of the ESP32 as an access point is to allow mobile devices (eg. your phone) to connect to the ESP32 and have a conversation with an application that runs upon it. See also: • esp_wifi_set_config • esp_wifi_set_mode

Working with connected stations

When our ESP32 is being an access point, we are saying that we wish to allow stations to connect to it. This brings in the story of managing those stations. Common things we might want to do are: • Determine when a new station connects • Determine when a previously connected station leaves • List the currently connected stations • Disconnect one or more currently connected stations We can register an event handler for detecting new station connects and existing station disconnects. The event handler will receive SYSTEM_EVENT_AP_STACONNECTED when a station connects and SYSTEM_EVENT_AP_STADISCONNECTED what a station leaves. We can get the list of currently connected stations using the esp_wifi_get_station_list() function. This returns a linked list of stations. The storage for this list is allocated for us and we should indicate that we are no longer in need of it by calling esp_wifi_free_station_list() when done. If for some reason the logic in our environment wants to forcibly disconnect a currently connected station, we can use the esp_wifi_kick_station() call. See also: • Handling WiFi events • esp_wifi_free_station_list • esp_wifi_get_station_list • esp_wifi_kick_station

WiFi at boot time

The ESP32 can store WiFi start-up information in flash memory. This allows it to perform its functions at start-up without having to ask the user for any special or additional information. This capability is controlled by a function called esp_wifi_set_auto connect() and its partner called esp_wifi_get_auto_connect(). The values of the settings used for an auto connect are those that are saved in flash memory. These are the values set when we call esp_wifi_set_config() but only if we have instructed the ESP32 to record those settings to flash. This is itself controlled by a call to esp_wifi_set_storage(). See also: • esp_wifi_set_auto_connect • esp_wifi_get_auto_connect • esp_wifi_set_storage

The DHCP client

When the ESP32 connects to an access point as a station, it also runs a DHCP client to connect to the DHCP server that it assumes is also available at the access point. From there, the station is supplied its IP address, gateway address and netmask. There are times however when we want to supply our own values for this data. We can do this by calling tcpip_adapter_set_ip_info() during setup. The recipe is as follows: tcpip_adapter_init(); tcpip_adapter_dhcpc_stop(); tcpip_adapter_set_ip_info(); esp_wifi_init(); esp_wifi_set_mode(); esp_wifi_set_config(); esp_wifi_start(); esp_wifi_config(); (Note that the parameters are omitted in the above). The setup for calling tcpip_adapter_set_ip_info() can be as follows: tcpip_adapter_ip_info_t ipInfo; IP4_ADDR(&ipInfo.ip, 192,168,1,99); IP4_ADDR(&ipInfo.gw, 192,168,1,1); IP4_ADDR(&ipInfo.netmask, 255,255,255,0); tcpip_adapter_set_ip_info(TCPIP_ADAPTER_IF_STA, &ipInfo); Alternative, using strings we have: tcpip_adapter_ip_info_t ipInfo; inet_pton(AF_INET, "192.168.1.99", &ipInfo.ip); inet_pton(AF_INET, "192.168.1.1", &ipInfo.gw); inet_pton(AF_INET, "255.255.255.0", &ipInfo.netmask); tcpip_adapter_set_ip_info(TCPIP_ADAPTER_IF_STA, &ipInfo); See also: • tcpip_adapter_set_ip_info • tcpip_adapter_dhcpc_start • tcpip_adapter_dhcpc_stop • tcpip_adapter_dhcpc_get_status • tcpip_adapter_dhcpc_option • inet_pton

The DHCP server

When the ESP32 is performing the role of an access point, it is likely that you will want it to also behave as a DHCP server so that connecting stations will be able to be automatically assigned IP addresses and learn their subnet masks and gateways. The DHCP server can be started and stopped within the device using the APIs called wifi_softap_dhcps_start() and wifi_softap_dhcps_stop(). The current status (started or stopped) of the DHCP server can be found with a call to wifi_softap_dhcps_status(). The default range of IP addresses offered by the DHCP server is 192.168.4.1 upwards. The first address becomes assigned to the ESP8266 itself. It is important to realize that this address range is not the same address range as your LAN where you may be working. The ESP8266 has formed its own network address space and even though they may appear with the same sorts of numbers (192.168.x.x) they are isolated and independent networks. If you start an access point on the ESP8266 and connect to it from your phone, don't be surprised when you try and ping it from your Internet connected PC and don't get a response. See also: • Error: Reference source not found

Current IP Address, netmask and gateway

Should we need it, we can query the environment for the current IP address, netmask and gateway. The values of these are commonly set for us by a DHCP server when we connect to an access point. The function called tcpip_adapter_get_ip_info() returns our current value. Since the ESP32 can have two IP interfaces (one for an access point and one for a station), we supply which interface we wish to retrieve. When we connect to an access point and have chosen to use DHCP, when we are allocated an IP address, an event is generated that can be used as an indication that we now have a valid IP address. See also: • Handling WiFi events • Error: Reference source not found • tcpip_adapter_get_ip_info

WiFi Protected Setup – WPS

The ESP8266 supports WiFi Protected Setup in station mode. This means that if the access point supports it, the ESP8266 can connect to the access point without presenting a password. Currently only the "push button mode" of connection is implemented. Using this mechanism, a physical button is pressed on the access point and, for a period of two minutes, any station in range can join the network using the WPS protocols. An example of use would be the access point WPS button being pressed and then the ESP8266 device calling wifi_wps_enable() and then wifi_wps_start(). The ESP8266 would then connect to the network. See also: • wifi_wps_enable • wifi_wps_start • wifi_set_wps_cb • Simple Questions: What is WPS (WiFi Protected Setup) • Wikipedia: WiFi Protected Setup

Designs for bootstrapping WiFi

Imagine that we have built a project using an ESP32 that wishes to be network connected. In order for that to happen, we want the ESP32 to connect to an existing access point. That's works, because the ESP32 can be a WiFi station. In order for the ESP32 to connect to an access point, it needs to know two important items. It needs to know which network to join (the SSID) and it will need to know the password to use to connect to that network as most networks require authentication. And there is the puzzle. If the ESP32 is brought to a physically new environment, how will it "know" which network to connect with and what password to use? We should assume that the ESP32 doesn't have a screen attached to it. If it did, we could prompt the user for the information. One solution is to have the ESP32 initially "be" an access point. If it were an access point then we could use our phone to connect with it, ask it what WiFi networks it can see, provide a password for the network and allow it to connect. while (not done) { if (we know our ssid and password) { attempt to connect to the access point; if (we succeeded in the connection) { return; } } become an access point ourselves; listen for incoming browser requests; wait for an SSID/password pair to be entered; } We also need to handle the case where we think we have an SSID and password used to connect to an access point but either those have changed or else we are in a foreign location. In that case we must also fall back to being an access point and await new instructions. We can use non-volatile storage to save our SSID and password. We may wish to save not just one SSID/password pair but perhaps save an ordered list. That way when we teach our device how to connect to an access point and then later teach it how to connect to another one, we might end up back at the first. For example, imagine using an ESP32 at home with one network and the same ESP32 at work with a different network. We may also want to save static interface information if we either don't have or don't want to use the services of a DHCP server when we start as a station. See also: • Non Volatile Storage

Working with TCP/IP

TCP/IP is the network protocol that is used on the Internet. It is the protocol that the ESP32 natively understands and uses with WiFi as the transport. Books upon books have already been written about TCP/IP and our goal is not to attempt to reproduce a detailed discussion of how it works, however, there are some concepts that we will try and capture. First, there is the IP address. This is a 32bit value and should be unique to every device connected to the Internet. A 32bit value can be thought of as four distinct 8bit values (4 x 8=32). Since we can represent an 8bit number as a decimal value between 0 and 255, we commonly represent IP addresses with the notation <number>.<number>.<number>.<number> for example 173.194.64.102. These IP addresses are not commonly entered in applications. Instead a textual name is typed such as "google.com" … but don't be misled, these names are an illusion at the TCP/IP level. All work is performed with 32bit IP addresses. There is a mapping system that takes a name (such as "google.com") and retrieves its corresponding IP address. The technology that does this is called the "Domain Name System" or DNS. When we think of TCP/IP, there are actually three distinct protocols at play here. The first is IP (Internet Protocol). This is the underlying transport layer datagram passing protocol. Above the IP layer is TCP (Transmission Control Protocol) which provides the illusion of a connection over the connectionless IP protocol. Finally there is UDP (User Datagram Protocol). This too lives above the IP protocol and provides datagram (connectionless) transmission between applications. When we say TCP/IP, we are not just talking about TCP running over IP but are in fact using this as a shorthand for the core protocols which are IP, TCP and UDP and additional related application level protocols such as DNS, HTTP, FTP, Telnet and more.

The Lightweight IP Stack – lwip

If we think of TCP/IP as a protocol then we can break up our understanding of networking into two distinct layers. One is the hardware layer that is responsible for getting a stream of 1's and 0's from one place to another. Common implementations for that include Ethernet, Token Ring and (yes … I'm dating myself now … dial-up modems). These are characterized by physical wires from your devices. WiFi is itself a transport layer. It deals with using radio waves as the communication medium of 1's and 0's between two points. The specification for WiFI is IEEE 802.11. Once we can transmit and receive data, the next level is organizing the data over that physical network and this is where TCP/IP comes into play. It provides the rules and governance of data transmission, addressing, routing, protocol negotiations and more. Typically, TCP/IP is implemented in software over the underlying physical transport mechanism. Think about this a moment. Imagine I said to you that I have a "magic box" and if you put something in that box, it will magically be transported to a different box. That is the analogy of physical transport. The software that is TCP/IP adds mechanisms above that. For example, imagine the box is only 6 inches wide. If you want to send me something through our boxes, you have to chop it up and send it in pieces. Your end of the box story handles that. My box will receive the parts and reassemble them for me. Parts may arrive in order and some parts may even get lost on route and have to be re-sent from the originals. The hardware (the boxes) have no idea how to achieve that. All they know is a piece of data in one end will hopefully arrive at the other … but not guaranteed. TCP/IP is a big protocol. It contains lots of parts. Fortunately it is well specified and has been implemented by many vendors over the last 45 years. Some of the implementations of the whole stack of TCP/IP parts have been written as open source and are distributed and maintained by the community. What this means is that if one has a new hardware layer, one can (in principle) lift an already written implementation of TCP/IP, map it to your hardware, compile it for your environment and you are good to go. This is actually much easier said than done … and fortunately for us, our friends at Espressif have done the work for us. One such open source implementation of a TCP/IP stack is called "The LightweightIPStack" which is commonly referred to as "lwIP". This can be read about in detail at its home page (see the references). As part of the distribution of the ESP-IDF, we have libraries that provide an implementation lwIP. It is lwIP that provides the ESP32 the following services: • IP • ICMP • IGMP • MLD • ND • UDP • TCP • sockets API • DNS Again, the good news is that the vast majority of lwIP is of no importance to us, ESP32 application designers and developers. It is vitally important … but important to the internal operation of ESP32 and not exposed to us as consumers. See also: • lwIP 2.0.0

TCP

A TCP connection is a bi-directional pipe through which data can flow in both directions. Before the connection is established, one side is acting as a server. It is passively listening for incoming connection requests. It will simply sit there for as long as needed until a connection request arrives. The other side of the connection is responsible for initiating the connection and it actively asks for a connection to be formed. Once the connection has been constructed, both sides can send and receive data. In order for the "client" to request a connection, it must know the address information on which the server is listening. This address is composed of two distinct parts. The first part is the IP address of the server and the second part is the "port number" for the specific listener. If we think about a PC, you may have many applications running on it, each of which can receive an incoming connection. Just knowing the IP address of your PC is not sufficient to address a connection to the correct application. The combination of IP address plus port number provides all the addressing necessary. As an analogy to this, think of your cell phone. It is passively sitting there until someone calls it. In our story your phone is the listener. The address that someone uses to form a connection is your phone number which is comprised of an area code plus the remainder. For example, a phone number of (817) 555-1234 will reach a particular phone. However the area code of 817 is for Fort Worth in Texas … calling that by itself is not sufficient to reach an individual … the full phone number is required. No we will look at how an ESP32 can set itself up as a listener for an incoming TCP/IP connection and this requires that we begin to understand the important "sockets" API.

TCP/IP Sockets

The sockets API is a programming interface for working with TCP/IP networking. It is probably the most familiar API for network programming. Sockets programming is familiar to programmers on Linux, Windows, Java and more. TCP/IP network flows come in two flavors … connection oriented over TCP and datagram oriented over UDP. The sockets API provides distinct patterns of calls for both styles. For TCP, a server is built by: 1. Creating a TCP socket 2. Associating a local port with the socket 3. Setting the socket to listen mode 4. Accepting a new connection from a client 5. Receive and send data 6. Close the client/server connection 7. Going back to step 4 For a TCP client, we build by: 1. Creating a TCP socket 2. Connecting to the TCP server 3. Sending data/receiving data 4. Close the connection Now let us break these up into code fragments that we can analyze in more depth. The header definitions for the sockets API can be found in <lwip/sockets.h>. For both the client and the server applications, the task of creating a socket is the same. It is an API call to the socket() function. int sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) The return from socket() is an integer handle that is used to refer to the socket. Sockets have lots of state associated with them, however that state is internal to the TCP/IP and sockets implementation and need not be exposed to the network programmer. As such, there is no need to expose that data to the programmer. We can think of calling socket() as asking the run-time to create and initialize all the data necessary for a network communication. That data is owned by the run-time and we are passed a "reference number" or handle that acts as a proxy to the data. When ever we wish to subsequently perform work on that network connection, we pass back in that handle that was previously issued to us and we can correlate back to the connection. This isolates and insulates the programmer from the guts of the implementation of TCP/IP and leaves us with a useful abstraction. When we are creating a server side socket, we want it to listen for incoming connection requests. To do this, we need to tell the socket which TCP/IP port number it should be listening upon. Note that we don't supply the port number a directly from an int/short value. Instead we supply the value as returned by the htons() function. What this function does is convert the number into what is called "network byte order". This is the byte order that has been chosen by convention to be that used for transmitting unsigned multi byte binary data over the internet. It's actual format is "big endian" which means that if we take a number such as 9876 (decimal) then it is represented in binary as 00100110 10010100 or 0x26D4 in hex. For network byte order, we first transmit 00100110 (0x26) followed by 10010100 (0xD4). It is important to realize that the ESP32 is a little endian native architecture which means that we absolutely must transform 2 byte and 4 byte numbers into network byte order (big endian). On a given device, only one application at a time can be using any given local port number. If we want to associate a port number with an application, such as our server application in this case, we perform a task called "binding" which binds (or assigns) the port number to the socket which in turn is owned by the application. struct sockaddr_in serverAddress; serverAddress.sin_family = AF_INET; serverAddress.sin_addr.s_addr = htonl(INADDR_ANY); serverAddress.sin_port = htons(portNumber); bind(sock, (struct sockaddr *)&serverAddress, sizeof(serverAddress)); With the socket now associated with a local port number, we can request that the runtime start listening for incoming connections. We do this by calling the listen() API. Before calling listen(), connections from clients would have been rejected with an indication to the client that there was nothing at the corresponding target address. Once we call listen(), the server will start accepting incoming client connections. The API looks like: listen(sock, backlog) The backlog is the number of connection requests that the run-time will listen for and accept before they are handed off to the application for processing. The way to think about this is imagine that you are the application and you can only do one thing at a time. For example, you can only be talking to one person at a time on the phone. Now imagine you have a secretary who is handling your incoming calls. When a call arrives and you are not busy, the secretary hands off the call to you. Now imagine that you are busy. At that time, the secretary answers the phone and asks the caller to wait. When you free up, she hands you the waiting call. Now let us assume that you are still busy when yet another client calls. She also tells this caller to wait. We are starting to build a queue of callers. And this is where the backlog concept comes into play. The backlog instructs the run-time how many calls can be received and asked to wait. If more calls arrive than our backlog will allow, the run-time rejects the call immediately. Not only does this prevent run-away resource consumption at the server, it also can be used as an indication to the caller that it may be better served trying elsewhere. Now from a server perspective, we are about ready to do some work. A server application can now block waiting for incoming client connections. The thinking is that a server application's purpose in life is to handle client requests and when it doesn't have an active client request, there isn't anything for it to do but wait for a request to arrive. While that is certainly one model, it isn't necessarily the only model or even the best model (in all cases). Normally we like our processors to be "utilized". Utilized means that while it has productive work it can do, then it should do it. If the only thing our program can do is service client calls, then the original model makes sense. However, there are certain programs that if they don't have a client request to immediately service, might spend time doing something else that is useful. We will come back to

that notion later on. For now, we will look at the accept() function call. When accept() is called, one of two things will happen. If there is no client connection immediately waiting for us, then we will block until such time in the future when a client connection does arrive. At that time we will wake up and be handed the connection to the newly arrived client. If on the other hand we called accept() and there was already a client connection waiting for us, we will immediately be handed that connection and we carry on. In both cases, we call accept() and are returned a connection to a client. The distinction between the cases is whether or not we have to wait for a connection to arrive. The API call looks like: struct sockaddr_in clientAddress; socklen_t clientAddressLength = sizeof(clientAddress); int clientSock = accept(sock, (struct sockaddr *)&clientAddress, &clientAddressLength); The return from accept() is a new socket (an integer handle) that represents the connection between the requesting client and the server. It is vital to realize that this is distinct from the server socket we created earlier which we bound to our server listening port. That socket is still alive and well and exists to continue to service further client connections. The newly returned socket is the connection for the conversation that was initiated by this single client. Like all TCP connections, the conversation is symmetric and bi-directional. This means that there is now no longer the notion of a client and server … both parties can send and receive as they would like at any time. If we wish to create a socket client, the story is similar. Again we create a socket() but this time there is no need for a bind()/listen()/accept() story. Instead we use the connect() API to connect to the target TCP/IP endpoint. For example: struct sockaddr_in serverAddress; serverAddress.sin_family = AF_INET; inet_pton(AF_INET, "192.168.1.200", &serverAddress.sin_addr.s_addr); serverAddress.sin_port = htons(9999); int rc = connect(sock, (struct sockaddr *)&serverAddress, sizeof(struct sockaddr_in)); See also: • Native byte order, endian and network byte order • socket • bind • listen • accept • send • recv • connect • Wikipedia – Berkeley Sockets • Beej's Guide to Network Programming

Handling errors

Most of the sockets APIs return an int return code. If this code is < 0 then an error has occurred. The nature of the error can be found using the global int called "errno". However, in a multitasking environment, working with global variables is not recommended. In the sockets area, we can ask a socket for the last error it encountered using the following code fragment: int espx_last_socket_errno(int socket) { int ret = 0; u32_t optlen = sizeof(ret); getsockopt(socket, SOL_SOCKET, SO_ERROR, &ret, &optlen); return ret; } The meanings of the errors can be compared against constants. Here is a table of constants used in the current FreeRTOS implementation: Symbol Value Description EPERM 1 Operation not permitted ENOENT 2 No such file or directory ESRCH 3 No such process EINTR 4 Interrupted system call EIO 5 I/O error ENXIO 6 No such device or address E2BIG 7 Arg list too long ENOEXEC 8 Exec format error EBADF 9 Bad file number ECHILD 10 No child processes EAGAIN 11 Try again ENOMEM 12 Out of memory EACCES 13 Permission denied EFAULT 14 Bad address ENOTBLK 15 Block device required EBUSY 16 Device or resource busy EEXIST 17 File exists EXDEV 18 Cross-device link ENODEV 19 No such device ENOTDIR 20 Not a directory EISDIR 21 Is a directory EINVAL 22 Invalid argument ENFILE 23 File table overflow EMFILE 24 Too many open files ENOTTY 25 Not a typewriter ETXTBSY 26 Text file busy EFBIG 27 File too large ENOSPC 28 No space left on device ESPIPE 29 Illegal seek EROFS 30 Read-only file system EMLINK 31 Too many links EPIPE 32 Broken pipe EDOM 33 Math argument out of domain of func ERANGE 34 Math result not representable EDEADLK 35 Resource deadlock would occur ENAMETOOLONG 36 File name too long ENOLCK 37 No record locks available ENOSYS 38 Function not implemented ENOTEMPTY 39 Directory not empty ELOOP 40 Too many symbolic links encountered EWOULDBLOCK EAGAIN 41 Operation would block ENOMSG 42 No message of desired type EIDRM 43 Identifier removed ECHRNG 44 Channel number out of range EL2NSYNC 45 Level 2 not synchronized EL3HLT 46 Level 3 halted EL3RST 47 Level 3 reset ELNRNG 48 Link number out of range EUNATCH 49 Protocol driver not attached ENOCSI 50 No CSI structure available EL2HLT 51 Level 2 halted EBADE 52 Invalid exchange EBADR 53 Invalid request descriptor EXFULL 54 Exchange full ENOANO 55 No anode EBADRQC 56 Invalid request code EBADSLT 57 Invalid slot EBFONT 59 Bad font file format ENOSTR 60 Device not a stream ENODATA 61 No data available ETIME 62 Timer expired ENOSR 63 Out of streams resources ENONET 64 Machine is not on the network ENOPKG 65 Package not installed EREMOTE 66 Object is remote ENOLINK 67 Link has been severed EADV 68 Advertise error ESRMNT 69 Srmount error ECOMM 70 Communication error on send EPROTO 71 Protocol error EMULTIHOP 72 Multihop attempted EDOTDOT 73 RFS specific error EBADMSG 74 Not a data message EOVERFLOW 75 Value too large for defined data type ENOTUNIQ 76 Name not unique on network EBADFD 77 File descriptor in bad state EREMCHG 78 Remote address changed ELIBACC 79 Can not access a needed shared library ELIBBAD 80 Accessing a corrupted shared library ELIBSCN 81 .lib section in a.out corrupted ELIBMAX 82 Attempting to link in too many shared libraries ELIBEXEC 83 Cannot exec a shared library directly EILSEQ 84 Illegal byte sequence ERESTART 85 Interrupted system call should be restarted ESTRPIPE 86 Streams pipe error EUSERS 87 Too many users ENOTSOCK 88 Socket operation on non-socket EDESTADDRREQ 89 Destination address required EMSGSIZE 90 Message too long EPROTOTYPE 91 Protocol wrong type for socket ENOPROTOOPT 92 Protocol not available EPROTONOSUPPORT 93 Protocol not supported ESOCKTNOSUPPORT 94 Socket type not supported EOPNOTSUPP 95 Operation not supported on transport endpoint EPFNOSUPPORT 96 Protocol family not supported EAFNOSUPPORT 97 Address family not supported by protocol EADDRINUSE 98 Address already in use EADDRNOTAVAIL 99 Cannot assign requested address ENETDOWN 100 Network is down ENETUNREACH 101 Network is unreachable ENETRESET 102 Network dropped connection because of reset ECONNABORTED 103 Software caused connection abort ECONNRESET 104 Connection reset by peer ENOBUFS 105 No buffer space available EISCONN 106 Transport endpoint is already connected ENOTCONN 107 Transport endpoint is not connected ESHUTDOWN 108 Cannot send after transport endpoint shutdown ETOOMANYREFS 109 Too many references: cannot splice ETIMEDOUT 110 Connection timed out ECONNREFUSED 111 Connection refused EHOSTDOWN 112 Host is down EHOSTUNREACH 113 No route to host EALREADY 114 Operation already in progress EINPROGRESS 115 Operation now in progress ESTALE 116 Stale NFS file handle EUCLEAN 117 Structure needs cleaning ENOTNAM 118 Not a XENIX named type file ENAVAIL 119 No XENIX semaphores available EISNAM 120 Is a named type file EREMOTEIO 121 Remote I/O error EDQUOT 122 Quota exceeded ENOMEDIUM 123 No medium found EMEDIUMTYPE 124 Wrong medium type Configuration settings Within the "menuconfig" there are some settings that relate to TCP/IP and can be found within the lwIP settings. The settings are: • Max number of open sockets – integer – CONFIG_LWIP_MAX_SOCKETS – This is the number of concurrently open sockets. The default is 4 and the maximum appears to be 16. • Enable SO_REUSEADDR – boolean – LWIP_SO_REUSE – Using select() Imagine that we have multiple sockets each of which may be the source of incoming data. If we try and read() data from a socket, we normally block until data is ready. If we did this, then if data becomes available on another socket, we wouldn't know. An alternative is to try and read data in a non-blocking fashion. This too would be useful but would require that we test each socket in turn in a busy or polling fashion. This too is not optimal. Ideally what we would like to do is block while watching multiple sockets simultaneously and wake up when the first one has something useful for us to do. See also: • select • The world of select()

Differences from "standard" sockets

Two header files that are commonly found in other sockets implementations are not part of the ESP-IDF definition. They are: • netinet/in.h • arpa/inet.h Despite not being present, no obvious issues have been found and it is assumed that the content normally contained within has been distributed across other headers.

UDP/IP Sockets

If we think of TCP as forming a connection between two parties similar to a telephone call, then UDP is like sending a letter through the postal system. If I were to send you a letter, I would need to know your name and address. Your address is needed so that the letter can be delivered to the correct house while your name ensure that it ends up in your hands as opposed to someone else who may live with you. In TCP/IP terms, the address is the IP address and the name is the port number. With a telephone conversation, we can exchange as much or as little information as we like. Sometimes I talk, sometimes you talk … but there is no maximum limit on how much information we can exchange in one conversation. With a letter however, there are only so many pages of paper that will fit in the envelopes I have at my disposal. The notion of the mail analogy is how we might choose to think about UDP. The acronym stands for User Datagram Protocol and it is the notion of the datagram that is akin to the letter. A datagram is an array of bytes that are transmitted from the sender to the receiver as a unit. The maximum size of a datagram using UDP is 64KBytes. No connection need be setup between the two parties before data starts to flow. However, there is a down side. The sender of the data will not be made aware of a receiver's failure to retrieve the data. With TCP, we have handshaking between the two parties that lets the sender know that the data was received and, if not, can automatically retransmit until it has been received or we decide to give up. With UDP, and just like a letter, when we send a datagram, we lose sight of whether or not it actually arrives safely at the destination. Now is a good time to come back to IP addresses and port numbers. We should start to be aware that on a PC, only one application can be listening upon any given port. For example, if my application is listening on port 12345, then no other application can also be listening on that same port … not your application nor another copy/instance of mine. When an incoming connection or datagram arrives at a machine, it has arrived because the IP address of the sent data matches the IP address of the device at which it arrived. We then route within the device based on port numbers. And here is where I want to clarify a detail. We route within the machine based on the pair of both protocol and port number. So for example, if a request arrives at a machine for port 12345 over a TCP connection, it is routed to the TCP application watching port 12345. If a request arrives at the same machine for port 12345 over UDP, it is routed to the UDP application watching port 12345. What this means is that we can have two applications listening on the same port but on different protocols. Putting this more formally, the allocation space for port numbers is a function of the protocol and it is not allowed for two applications to simultaneously reserve the same port within the same protocol allocation space. Although I used the story of a PC running multiple applications, in our ESP32 the story is similar even though we just run one application on the device. If your single application should need to listen on multiple ports, don't try and use the same port with the same protocol as the second function call will find the first one has already allocated the port. This is a detail that I am happy for you to forget as you will rarely come across it but I wanted to catch it here for completeness. To program with UDP, once again we use sockets. To set up a socket server using UDP again we call socket() to create a socket and again we call bind() to specify the port number we wish to listen upon. There is no need for a call to listen(). When the server is ready to receive an incoming request, we call recvfrom() which blocks until a datagram is received. Once one arrives, we wake up and can process the request. The request contains a return address and we can send a response using sendto() should we wish. On the client side, we create a socket() and then can invoke sendto(). The call to sendto() takes the IP address and port of the target as parameters as well as the payload data. For example: int socket_fd = socket(AF_INET, SOCK_DGRAM, 0); sendto(socket_fd, data, size, 0, destAddr, destAddrLen); • socket • sendto • recvfrom

TLS, SSL and security

So far we have been thinking about making sockets calls that form a network connection and then sending and receiving data over that connection. However we have a security problem. The data that flows over the wire is not encrypted. This means that if one were to "sniff" or otherwise examine the network data, we would see the content of the data being transmitted. For example, if I send a password used for authentication, if we were to examine the content of the data, we would be able to determine the password I am using. It actually isn't that difficult to sense the data being sent and received. Excellent tools such as wireshark are used for debugging and can easily be used to examine the content of the network packets or stream. Obviously we are already exchanging credit card data, email and other sensitive information over the Internet so how is that done? The answer is a concept called the "Secure Socket Layer" or SSL. SSL provides the capability to encrypt the data before transmission such that only the intended recipient can decrypt it. Conversely, any responses sent by the recipient are also encrypted such that only we can decrypt the data. If someone were to capture or otherwise examine the data being sent over the wire, there is no way for them to get back to the original data content. The way this works is through the concept of private keys. Imagine I think of a very large random number (and by large I mean VERY large). We call this private number my private key. Now imagine that associated with the private key is a corresponding number (the public key) that can be used to decrypt a message that was encoded using the private key. Now imagine I want to correspond with a partner securely. I send a request (unencrypted) to the partner and ask for his public key. He sends that back and I send to him a copy of MY public key encrypted with his public key. Since only a matching pair of public/private keys can be used to decrypt data, only the desired recipient can decrypt the message at which point he will have a copy of my public key. Now in the future I can send him messages encrypted with my private key and further encrypted with his public key and he will be able to decode them with his copy of his private key and my public key while he can send me encrypted messages encoded with his private key which I can decode with my copy of his public key. By having exchanged public keys, we are now good to continue exchanging data without fear that it will be seen by anyone else. All of this encryption of data happens outside and above the knowledge of TCP/IP networking. TCP/IP provides the delivery of data but cares nothing about its content. As such, and at a high level, if we wish to exchange secure data, we must perform the encryption and decryption using algorithms and libraries that live outside of the sockets API and use sockets as the transport for transmitting and receiving the encrypted data that is fed into and received from the encryption algorithms. When using mbed TLS, we need a large stack size. I don't yet know how small we can get away with but I have been using 8000 bytes. See also: • mbed TLS • mbed TLS home page • mbed TLS tutoria l • mbed TLS API reference

mbedTLS app structure

Let us start to break down the structure of a TLS application that uses the mbedTLS APIs. First there is the notion of a network context that is initialized by a call to mbedtls_net_init(). mbedtls_net_context server_fd; mbedtls_net_init(&server_fd); There is nothing more to explain here. The data that is initialized is "opaque" to us and the invocation of this function is part of the rules. next comes the initialization of the SSL context with a call to mbedtls_ssl_init(). mbedtls_ssl_context ssl; mbedtls_ssl_init(&ssl); Again, there is nothing more to explain here. The data is again opaque and this function merely initializes it for us. Calling this function is also part of the rules. Now we call mebtls_ssl_config_init(). mbedtls_ssl_config config; mbedtls_ssl_config_init(&config); These initializations repeat for other data types including: mbedtls_ctr_drbg_context ctr_drbg; mbedtls_ctr_drbg_init(&ctr_drbg); mbedtls_entropy_context entropy; mbedtls_entropy_init(&entropy); mbedtls_x509_crt cacert; mbedtls_x509_crt_init(&cacert); SSL utilizes good random number generators. What is a "good" random number? Since computers are deterministic devices, the generation of a random number is performed through the execution of an algorithm and since algorithms are deterministic, then a sequence of numbers generated by these functions might, in principle, be predictable. A good random number generator is one where the sequence of numbers produced is not at all easily predictable and generates values with no biases towards their values with an equal probability of any number within a range being chosen. We initialize the random number generator with a call to mbedtls_ctr_drbg_seed(). Note: We see the phrase "ctr_drbg" … that is an acronym for "Counter mode Deterministic Random Byte Generator". It is an industry standard specification/algorithm for generating random numbers. http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-90Ar1.pdf With the setup under our belts, it is now time to start considering SSL based communication. Since we are considering SSL over sockets, if we were not using TLS sockets, we would perform a call to socket() to create a socket and then connect() to connect to our partner. In the world of mbedtls, we call mbedtls_net_connect(). This has the form: mbedtls_net_connect(&server_fd, <hostname>, <port>, MBEDTLS_NET_PROTO_TCP); The hostname and port define where we are connecting to. Notice the first parameter. This is the mbedtls_net_context structure that we initialized with a call to mbedtls_net_init() previously. We should always check the return code to ensure that the connection was successful. Now we get to configure our SSL defaults with a call to mbedtls_ssl_config_defaults(). For example: mbedtls_ssl_config_defaults( &conf, MBEDTLS_SSL_IS_CLIENT, MBEDTLS_SSL_TRANSPORT_STREAM, MBED_SSL_PRESET_DEFAULT) When we are communicating via SSL, we commonly wish to validate that the credentials provided by our partner indicate that they are who they claim to be. This process is called authentication. We can define what kind of authentication we wish to perform by calling mbedtls_ssl_conf_authmode(). mbedtls_ssl_conf_authmode(&ssl, MBEDTLS_SSL_VERIFY_NONE) Earlier we said that SSL is heavily dependent on a good random number generator. Now we tell the environment which random number generator we wish to use: mbedtls_ssl_conf_rng(&ssl, mbedtls_ctr_drbg_random, &ctr_drbg) Next we do some more SSL context setup by calling mbedtls_ssl_set_hostname(). mbedtls_ssl_set_hostname(&ssl, "name") Now we instruct the SSL environment which functions to use to send and receive data by calling mbedtls_ssl_set_bio(). mbedtls_ssl_set_bio(&ssl, &server_fd, mbedtls_net_send, mbedtls_net_recv, NULL) At this point, we have formed a connection to our partner and configured the SSL environment. What remains is to actually read and write data. To write data we call mbed_ssl_write(). mbedtls_ssl_write(&ssl, buf, len) and to read data we call mbedtls_ssl_read(). mbedtls_ssl_read(&ssl, buf, len) See also: • mbedtls_net_init • mbedtls_ssl_init • mbedtls_ssl_config_init • mbedtls_net_connect • mbedtls_ssl_config_defaults • mbedtls_ssl_conf_authmode • mbedtls_ssl_conf_rng • mbedtls_ssl_set_hostname • mbedtls_ssl_set_bio • mbedtls_ssl_write • mbedtls_ssl_read

mbedTLS Example

Here is a sample function that has been tested on an ESP32 to make an HTTPS call to an HTTPS server to retrieve some results.

1. include "mbedtls/platform.h"
2. include "mbedtls/ctr_drbg.h"
3. include "mbedtls/debug.h"
4. include "mbedtls/entropy.h"
5. include "mbedtls/error.h"
6. include "mbedtls/net.h"
7. include "mbedtls/ssl.h"
8. include "esp_log.h"
9. include "string.h"
10. include "stdio.h"
11. define SERVER_NAME "httpbin.org"
12. define SERVER_PORT "443"

static char tag[] = "callhttps"; static char errortext[256]; static void my_debug(void *ctx, int level, const char *file, int line, const char *str) { ((void) level); ((void) ctx); printf("%s:%04d: %s", file, line, str); } void callhttps() { ESP_LOGD(tag, "--> callhttps\n"); mbedtls_net_context server_fd; mbedtls_entropy_context entropy; mbedtls_ctr_drbg_context ctr_drbg; mbedtls_ssl_context ssl; mbedtls_ssl_config conf; mbedtls_x509_crt cacert; int ret; int len; char *pers = "ssl_client1"; unsigned char buf[1024]; mbedtls_net_init(&server_fd); mbedtls_ssl_init(&ssl); mbedtls_ssl_config_init(&conf); mbedtls_x509_crt_init(&cacert); mbedtls_ctr_drbg_init(&ctr_drbg); mbedtls_entropy_init(&entropy); mbedtls_ssl_conf_dbg(&conf, my_debug, stdout); mbedtls_debug_set_threshold(2); // Log at error only ret = mbedtls_ctr_drbg_seed( &ctr_drbg, mbedtls_entropy_func, &entropy, (const unsigned char *) pers, strlen(pers)); if (ret != 0) { ESP_LOGE(tag, " failed\n ! mbedtls_ctr_drbg_seed returned %d\n", ret); return; } ret = mbedtls_net_connect(&server_fd, SERVER_NAME, SERVER_PORT, MBEDTLS_NET_PROTO_TCP); if (ret != 0) { ESP_LOGE(tag, " failed\n ! mbedtls_net_connect returned %d\n\n", ret); return; } ret = mbedtls_ssl_config_defaults( &conf, MBEDTLS_SSL_IS_CLIENT, MBEDTLS_SSL_TRANSPORT_STREAM, MBEDTLS_SSL_PRESET_DEFAULT); if (ret != 0) { ESP_LOGE(tag, " failed\n ! mbedtls_ssl_config_defaults returned %d\n\n", ret); return; } mbedtls_ssl_conf_authmode(&conf, MBEDTLS_SSL_VERIFY_NONE); mbedtls_ssl_conf_rng(&conf, mbedtls_ctr_drbg_random, &ctr_drbg); ret = mbedtls_ssl_setup(&ssl, &conf); if (ret != 0) { mbedtls_strerror(ret, errortext, sizeof(errortext)); ESP_LOGE(tag, "error from mbedtls_ssl_setup: %d - %x - %s\n", ret, ret, errortext); return; } ret = mbedtls_ssl_set_hostname(&ssl, "httpbin.org"); if (ret != 0) { mbedtls_strerror(ret, errortext, sizeof(errortext)); ESP_LOGE(tag, "error from mbedtls_ssl_set_hostname: %d - %x - %s\n", ret, ret, errortext); return; } mbedtls_ssl_set_bio(&ssl, &server_fd, mbedtls_net_send, mbedtls_net_recv, NULL); char *requestMessage = \ "GET /ip HTTP/1.1\r\n" \ "User-Agent: kolban\r\n" \ "Host: httpbin.org\r\n" \ "Accept-Language: en-us\r\n" \ "Accept-Encoding: gzip, deflate\r\n" \ "\r\n"; sprintf((char *)buf, requestMessage); len = strlen((char *)buf); ret = mbedtls_ssl_write(&ssl, buf, len); if (ret < 0) { mbedtls_strerror(ret, errortext, sizeof(errortext)); ESP_LOGE(tag, "error from write: %d -%x - %s\n", ret, ret, errortext); return; } len = sizeof(buf); ret = mbedtls_ssl_read(&ssl, buf, len); if (ret < 0) { ESP_LOGE(tag, "error from read: %d\n", len); return; } printf("Result:\n%.*s\n", len, buf); mbedtls_net_free(&server_fd); mbedtls_ssl_free(&ssl); mbedtls_ssl_config_free(&conf); mbedtls_ctr_drbg_free(&ctr_drbg); mbedtls_entropy_free(&entropy); ESP_LOGV(tag, "All done"); } Notes: When debugging MBED in Curl set MBEDTLS_DEBUG to 1 in curl_config.h

OpenSSL

OpenSSL is a popular implementation of an SSL stack. In the ESP32 environment, the selected stack for SSL/TLS is mbedTLS which is not the same as OpenSSL. As part of the ESP-IDF, a mapping layer has been provided that exposes the OpenSSL API on top of an mbedTLS implementation.

Name Service

On the Internet, server machines can be found by their Domain Name Service (DNS) names. This is the service that resolves a human readable representation of a machine such as "google.com" into the necessary IP address value (eg. 216.58.217.206). In order for this transformation to happen, the ESP32 needs to know the IP address of one or more DNS servers that it will then use to perform the name to IP address mapping. If we are using DHCP then nothing else need be done as the DHCP server automatically provides the DNS server addresses. However, if we should not be using DHCP (for example we are using static IP addresses), then we need to instruct the ESP32 of the locations of the DNS servers manually. We can do this using dns_setserver() function. This takes an IP address as input along with which of the two possible DNS servers to use. The ESP32 is configured to know the identity of up to two external name servers. The reason for two is that if an attempt to reach the first one fails, we will utilize the second one. We can retrieve our current DNS server identities using dns_getserver(). Google publicly makes available two name servers with the addresses of 8.8.8.8 and 8.8.4.4. Once we have define the name servers, we can look up the address of a host name using the gethostbyname() function. During development, we may wish to test a specific DNS server to validate that it can resolve a host name. An excellent Linux tool to perform this task is "nslookup". It has many options but for our purposes, we can supply it the host name to lookup and the DNS server to use: $ nslookup example.com 8.8.8.8 Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: Name: example.com Address: 93.184.216.34 See also: • gethostbyname • dns_getserver • dns_setserver • Wikipedia: Domain Name System • Google: Public DNS

Multicast Domain Name Systems

On a local area network with dynamic devices coming and going, we may want one device to find the IP address of another device so that they may interact with each other. The problem though is that IP addresses can be dynamically allocated by a DHCP server running on a WiFi access point. This means that the IP address of a

device is likely not going to be static. In addition, it is not a great usability story to refer to devices by their IP addresses. What we need is some form of dynamic name service for finding devices by name where their IP addresses aren't administrator configured. This is where the Multicast Domain Name System (mDNS) comes into play. At a high level, when a device wishes to find another device with a given name, it broadcasts a request to all members of the network asking for a response from the device that has that name. If a machine believes it has that identity, it responds with its own broadcast which includes its name and IP address. Not only does this satisfy the original request, but other machines on the network can see this interaction and cache the response for themselves. This means that should they need to resolve the same host in the future, they already have the answer. Using the Multicast Domain Name System (mDNS) an ESP32 can attempt to resolve a host name of a machine on the local network to its IP address. It does this by broadcasting a packet asking for the machine with that identity to respond. The name service demons are implemented by Bonjour and nss-mdns (Linux). Normally, hosts located using this technique belong to a domain ending in ".local". To determine if your PC is participating in mDNS you can examine whether or not it is listening on UDP port 5353. This is the port used for mDNS communications. See also: • Wikipedia – Multicast DNS • IETF RFC 6762: Multicast DNS • Multicast DNS • New DNS Technologies in the Lan • Avah i – Implementation of mDNS … source project for Unix machines • Adafruit – Bonjour (Zeroconf) Networking for Windows and Linux • chrome.mdns – API description for Chrome API for mDNS • Android – ZeroConf Browser

mDNS API programming

The ESP-IDF provides a set of rich APIs for programming mDNS on the ESP32. Specifically, we can either advertise ourselves in mDNS or else query existing mDNS information. The attributes of an mDNS server entry appear to be: • hostname – mdns_set_hostname() • default instance – mdns_set_instance() • service – mdns_service_add() ◦ type – _http, _ftp etc etc ◦ protocol – _tcp, _udp etc etc ◦ port – port number • instance name for service – mdns_service_instance_set() • TXT data for service – mdns_service_txt_set() See also: • mdns_set_hostname • mdns_set_instance • mdns_service_add • mdns_service_instance_set • mdns_service_port_set

Installing Bonjour

Launch the Bonjour installer: If all has gone well, we will find a new Windows service running called "Bonjour Service": There is also a Bonjour browser available here … http://hobbyistsoftware.com/bonjourbrowser

Avahi

An implementation of Multicast DNS on Linux is called Avahi. Avahi runs as the systemd daemon called "avahi-daemon". We can determine whether or not it is running with: $ systemctl status avahi-daemon ● avahi-daemon.service - Avahi mDNS/DNS-SD Stack Loaded: loaded (/lib/systemd/system/avahi-daemon.service; enabled) Active: active (running) since Wed 2016-01-20 22:13:35 CST; 1 day 13h ago Main PID: 384 (avahi-daemon) Status: "avahi-daemon 0.6.31 starting up." CGroup: /system.slice/avahi-daemon.service ├─384 avahi-daemon: running [raspberrypi.local] └─426 avahi-daemon: chroot helper The avahi-daemon utilizes a configuration file found at /etc/avahi/avahi-daemon.conf. The default name that avahi advertizes itself as is the local hostname. When host-name resolution is performed, the system file called /etc/nsswitch.conf is used to determine the order of resolution. Specifically the hosts entry contains the name resolution. An example would be: hosts: files mdns4_minimal [NOTFOUND=return] dns Which says "first look in /etc/hosts, then consult mDNS and then use full DNS". What this means is that a device which advertizes itself with mDNS can be found via a lookup of "<hostname>.local". For example, if I boot up a Linux machine which gets a dynamic IP address through DHCP and the hostname of that machine is "chip1", then I can reach it with a domain name address of "chip1.local". If the IP address of the device changes, subsequent resolutions of the domain name will continue to correctly resolve. Avahi tools are not installed by default but can be installed using the "avahi-utils" package: $ sudo apt-get install avahi-utils To see the list of mDNS devices in your network, we can use the avahi-browse command. For example: $ avahi-browse -at + wlan1 IPv6 chip1 [ce:79:cf:21:db:95] Workstation local + wlan1 IPv4 chip1 [ce:79:cf:21:db:95] Workstation local + wlan0 IPv6 pizero [00:36:76:21:97:a3] Workstation local + wlan0 IPv6 raspi3 [b8:27:eb:9d:fc:60] Workstation local + wlan0 IPv6 chip1 [cc:79:cf:21:db:95] Workstation local + wlan0 IPv4 pizero [00:36:76:21:97:a3] Workstation local + wlan0 IPv4 raspi3 [b8:27:eb:9d:fc:60] Workstation local + wlan0 IPv4 chip1 [cc:79:cf:21:db:95] Workstation local + wlan0 IPv6 pizero Remote Disk Management local + wlan0 IPv6 raspi3 Remote Disk Management local + wlan0 IPv4 raspi3 Remote Disk Management local + wlan0 IPv4 pizero Remote Disk Management local + wlan0 IPv4 WDMyCloud Apple File Sharing local + wlan0 IPv4 WDMyCloud _wd-2go._tcp local + wlan0 IPv4 WDMyCloud Web Site local + wlan0 IPv4 Living Room _googlecast._tcp local + wlan0 IPv4 123456789 _teamviewer._tcp local To access an mDNS advertized server from Microsoft Windows, you will need a service similar to Apple's Bonjour installed. Bonjour is distributed as part of Apple's iTunes product. Once installed, we should be able to access the published servers at their <name>.local address. A partner windows tool called "Bonjour Browser" is available which displays an mDNS listing of servers on windows. See also: • avahi home page • man(1) – avahi-browse • man(5) – avahi-daemon.conf

Working with SNTP

SNTP is the Simple Network Time Protocol and allows a device connected to the Internet to learn the current time. In order to use this, you must know of at least one time server located on the Internet. The US National Institute for Science and Technology (NIST) maintains a number of these which can be found here: • http://tf.nist.gov/tf-cgi/servers.cg i Other time servers can be found all over the globe and I encourage you to Google search for your nearest or country specific server. Once you know the identity of a server by its host name or IP address, you can call either of the functions called sntp_setservername() or sntp_setserver() to declare that we wish to use that time server instance. The ESP32 can be configured with up to three different time servers so that if one or two are not available, we might still get a result. The ESP32 must also be told the local timezone in which it is running. This is set with a call to sntp_set_timezone() which takes the number of hours offset from UTC. For example, I am in Texas and my timezone offset becomes "-5". Although this function is present, I would suggest using the POSIX tzset() function instead. With these configured, we can start the SNTP service on the ESP32 by calling sntp_init(). This will cause the device to determine its current time by sending packets over the network to the time servers and examining their responses. It is important to note that immediately after calling sntp_init(), you will not yet know what the current time may be. This is because it may take a few seconds for the ESP32 to sends the time requests and get their responses and this will all happen asynchronously to your current commands and won't complete till sometime later. When ready, we can retrieve the current time with a call to sntp_get_current_timestamp() which will return the number of seconds since the 1st of January 1970 UTC. We can also call the function called sntp_get_real_time() which will return a string representation of the time. While these functions obviously exist, I would not recommend using them. Instead look at the POSIX alternatives which are time() and asctime(). Here is an example of using SNTP to set the time: ip_addr_t addr; sntp_setoperatingmode(SNTP_OPMODE_POLL); inet_pton(AF_INET, "129.6.15.28", &addr); sntp_setserver(0, &addr); sntp_init(); The time can be accessed from a variety of POSIX compliant functions including: • asctime – Build a string representation of time. • clock – Return processor time. • ctime – Build a string representation of time. • difftime – Calculate a time difference. • gettimeofday – Retrieve the current time of day. • gmtime – Produce a struct tm from a time_t. • localtime – Produce a struct tm from a time_t. • settimeofday – Set the current time. • strftime – Format a time_t to a string. • time – Get the current time as a time_t (seconds since epoch). See also: • SNTP API • Timers and time • asctime • ctime • gmtime • localtime • strftime • IETF RFC5905: Network Time Protocol Version 4: Protocol and Algorithms Specification

Java Sockets

The sockets API is the defacto standard API for programming against TCP/IP. My programming language of choice is Java and it has full support for sockets. What this means is that I can write a Java based application that leverages sockets to communicate with the ESP32. I can send and receive data through quite easily. In Java, there are two primary classes that represents sockets, those are java.net.Socket which represents a client application which will form a connection and the second class is java.net.ServerSocket which represents a server that is listening on a socket awaiting a client connection. Since the ESP32 can be either a client or a server, both of these Java classes will come into play. To connect to an ESP32 running as a server, we need to know the IP address of the device and the port number on which it is listening. Once we know those, we can create an instance of the Java client with: Socket clientSocket = new Socket(ipAddress, port); This will form a connection to the ESP32. Now we can ask for both an InputStream from which to receive partner data and an OutputStream to which we can write data. InputStream is = clientSocket.getInputStream(); OutputStream os = clientSocket.getOutputStream(); When we are finished with the connection, we should call close() to close the Java side of the connection: clientSocket.close(); It really is as simple as that. Here is an example application: package kolban; import java.io.OutputStream; import java.net.Socket; import org.apache.commons.cli.CommandLine; import org.apache.commons.cli.CommandLineParser; import org.apache.commons.cli.DefaultParser; import org.apache.commons.cli.Options; public class SocketClient { private String hostname; private int port; public static void main(String[] args) { Options options = new Options(); options.addOption("h", true, "hostname"); options.addOption("p", true, "port"); CommandLineParser parser = new DefaultParser(); try { CommandLine cmd = parser.parse(options, args); SocketClient client = new SocketClient(); client.hostname = cmd.getOptionValue("h"); client.port = Integer.parseInt(cmd.getOptionValue("p")); client.run(); } catch (Exception e) { e.printStackTrace(); } } public void run() { try { int SIZE = 65000; byte data[] = new byte[SIZE]; for (int i = 0; i < SIZE; i++) { data[i] = 'X'; } Socket s1 = new Socket(hostname, port); OutputStream os = s1.getOutputStream(); os.write(data); s1.close(); System.out.println("Data sent!"); } catch (Exception e) { e.printStackTrace(); } } } // End of class // End of file To configure a Java application as a socket server is just as easy. This time we create an instance of the SocketServer class using: SocketServer serverSocket = new SocketServer(port) The port supplied is the port number on the machine on which the JVM is running that will be the endpoint of remote client connection requests. Once we have a ServerSocket instance, we need to wait for an incoming client connection. We do this using the blocking API method called accept(). Socket partnerSocket = serverSocket.accept(); This call blocks until a client connect arrives. The returned partnerSocket is the connected socket to the partner which can used in the same fashion as we previously discussed for client connections. This means that we can request the InputStream and OutputStream objects to read and write to and from the partner. Since Java is a multithreaded language, once we wake up from accept() we can pass off the received partner socket to a new thread and repeat the accept() call for other parallel connections. Remember to close() any partner socket connections you receive when you are done with them. So far, we have been talking about TCP oriented connections where once a connection is opened it stays open until closed during which time either end can send or receive independently from the other. Now we look at datagrams that use the UDP protocol. The core class behind this is called DatagramSocket. Unlike TCP, the DatagramSocket class is used both for clients and servers. First, let us look at a client. If we wish to write a Java UDP client, we will create an instance of a DatagramSocket using: DatagramSocket clientSocket = new DatagramSocket(); Next we will "connect" to the remote UDP partner. We will need to know the IP address and port that the partner is listening upon. Although the API is called "connect", we need to realize that no connection is formed. Datagrams are connectionless so what we are actually doing is associating our client socket with the partner socket on the other end so that when we actually wish to send data, we will know where to send it to. clientSocket.connect(ipAddress, port); Now we are ready to send a datagram using the send() method: DatagramPacket data = new DatagramPacket(new byte[100], 100); clientSocket.send(data); To write a UDP listener that listens for incoming datagrams, we can use the following: DatagramSocket serverSocket = new DatagramSocket(port); The port here is the port number on the same machine as the JVM that will be used to listen for incoming UDP connections. To wait for an incoming datagram, call receive(). DatagramPacket data = new DatagramPacket(new byte[100], 100); clientSocket.receive(data); If you are going to use the Java Socket APIs, read the JavaDoc thoroughly for these classes are there are many features and options that were not listed here. See also: • Java tutorial: All About Sockets • JDK 8 JavaDoc

Bluetooth

The ESP32 has native Bluetooth support (version 4.2). This means that it can interact with Bluetooth devices such as keyboards, mice and cell phones. Let us review what Bluetooth means for us. Bluetooth is a wireless communication protocol/technology that provides data transfer over a radio signal. Let us assume that ESP32 is one end of the connection and any other Bluetooth device can be at the other. For security purposes, arbitrary Bluetooth devices can't simply be "used" without some explicit authorization. For example, it would be very wrong if I could bring my Bluetooth headset near your phone and start listening to your calls. To achieve security, a process called "pairing" needs to be performed. This achieves a level of trust between the two Bluetooth devices such that they subsequently allow connection without having to be re-paired.

Bluetooth specification

Bluetooth is a specification for wireless communication between multiple electronic devices. At present, there are two primary standards … these are Bluetooth (Classic) and Bluetooth LE. The "LE" stands for Low Energy and is the specification for devices that wish to be powered by simple batteries and yet have sufficient life span. In the Bluetooth story, we have devices which are "masters" and devices which are "slaves". A slave can only form a connection and communicate with a master while a master can form concurrent connections with multiple slaves. One slave can not directly communicate with another slave. Instead it must communicate with the master and the master relay the request. The simplest communication is one master and one slave but if we have multiple slaves connected to the same master, the resulting "network" is termed a "piconet". Each device that participates in the conversation has a unique address that is a 48 bit value commonly written as 12 hex values (6 bytes). This address is known as the "Bluetooth Device Address" and may be seen in other documentation abbreviated to "BD_ADDR". The encoding of a Bluetooth address is that the first 24 bits encode the organization responsible for allocating the remaining address and the remaining 24 bits are the address itself. However the full 48 bits are the complete identity of the device. As well as having a unique address, each device can have a symbolic name to help us meaningfully identify it. This is termed the display name. The display name is only a mapping to a Bluetooth address and it is really the address that is used to distinguish one device from another. It is also important to note that there are no uniqueness constraints on device names. Multiple devices may select the same device name. Let us assume that initially, we have two devices and neither of them know about the other. What must now happen is a discovery process. One of them will broadcast an "inquiry" request. Devices receiving the inquiry can respond with their own existence by transmitting their own address and possibly additional information. It is common that the response to an inquiry does not contain the display name of the responding device. If the display name is needed, the inquirer can transmit a directed request now that it knows the address of the responding device which will solicit the name as a further response. A device does not have to respond to an inquiry request. It has a property setting called an "inquiry scan" that controls whether or not it responds to such. If the attribute is on, then it will respond to an inquiry request and if off, then it will not respond. Think of the phrase "inquiry scan" as the devices choice as to whether it performs the action of "scanning for inquires". Once the two devices know each other's addresses, a connection can be formed between them through a process known as "paging". Again, a device does not have to service a received connection request. It has a property setting called a "paging scan" which controls this. If on, then a paging request causes a new connection to be formed. If off, it will not accept a new connection request. Once a connection is formed between the devices, that connection can be maintained in a variety of states, the most common being active. However other states are available and are used to save power when there is no active communication of data anticipated. In order to permit devices to communicate with each other, there has to be an element of security involved. We usually don't want arbitrary devices to be able to connect with each other and share arbitrary information. To achieve this, a process called "bonding" is enacted. Bonding is achieved through the notion of "pairing". In pairing, the devices exchange their addresses, names and other data and generate keys that they share with each other. Pairing typically requires an explicit interaction from the user to permit the pairing to succeed. The user interaction can be as simple as "I approve this device" with a button click or it can be richer with the entry of a pass-code to authenticate and prove that one is who one claims to be. The Bluetooth protocol provides support for different classes of power. This translates directly into the signal strength of the radio. Remember that the more power used by the radio, the heavier the drain on the power source. If the power source is batteries, the more power used to transmit data, the shorter the life of the batteries. At the lower levels, Bluetooth takes care of exchanging data between partners. However, there is much more to Bluetooth than just simple data exchange. In order to provide interoperability between devices built by multiple manufacturers, higher level protocols called "profiles" have been defined. These profiles define "what" is transmitted over Bluetooth for a given device function. Some of the profiles we will come across include: • HSP – Head Set Profile (eg. a Bluetooth ear piece). • HFP – Hands Free Profile (eg. Bluetooth communication in a car). • HID – Human Interface Device (eg. a keyboard or mouse). • SPP – Serial Port Profile. • A2DP – Advanced Audio Distribution profile (eg. connection to Bluetooth speakers). • AVRCP – Audio Visual Remote Control Profile. Knowing that these protocols exist, it is not sufficient that two Bluetooth devices are in range of each other, they must also both support the same profile that is desired to be used. At a higher level that the connection are the transport protocols available to us. These include: • RFCOMM – Radio Frequency Communications Protocol. This protocol provides a reliable stream oriented transmission. Loosely, you can compare it to TCP. • L2CAP – Logical Link Control and Adaption Protocol. This is a packet oriented protocol. RFCOMM builds on L2CAP. • ACL – Asynchronous Connection oriented Logical protocol. L2CAP builds on ACL. • SCO – Voice quality audio protocol. To allow multiple conversations to be processed in parallel, the concept of the "port" is introduced. This is similar to a TCP/IP port number. L2CAP ports can be odd numbered values between 1 and 32767. For RFCOMM, the port numbers are called channels and are between 1 and 30. In the Bluetooth documentation, ports are referred to as "Protocol Service Multiplexers" or "PSMs". Certain port numbers used by L2CAP are designated as reserved for well defined purposes. See also: • Introduction to Bluetooth Low Energy

Bluetooth UUIDs

A UUID is a 16 byte number (128 bits). They are commonly written in hexadecimal with 1 byte corresponding to 2 hex digits. The most common written format is 4-2-2-2-6. XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX When one receives a UUID, it is the value of the UUID that is used to describe the nature of the service or data. Each different type of service will have its own unique UUID. To reduce transmission overhead, the specification defines some well described services. If one implements one of those services, then we don't need to transmit the whole 128 bit UID but can get away with less. A special UUID of the form: XXXXXXXX-0000-1000-8000-00805F9BE4FB is available where only the first 4 bytes (32 bits) are needed to identify the type of service.

Bluetooth GAP

The Generic Access Protocol (GAP). It is GAP that determine which devices can interact with each other. In the GAP story there are primarily two classes of "things". There is the central device and the peripheral device. We can send advertizing using the Advertizing Data payload or the Scan Response payload. A peripheral device will constantly transmit its advertizing payload which can contain up to 31 bytes of data. The peripheral transmits its advertizing data every advertizing interval period. The Advertizing Data payload is present in all BT devices. If the central receiver of the advertizing data payload wishes, it can request a scan response and the peripheral will send back a scan response payload. When a peripheral is transmitting advertizing data, we can think of it as being in the mode of broadcasting. It is transmitting its data which may or may not be seen by a corresponding observer. The concept of BLE advertizing is powerful beyond just finding devices to form subsequent connections. The advertizing packets can contain data in their payload. If the data being transmitted does not need to be secure (for example, the outside temperature), then we have the core of an interesting solution in its own right. The peripheral can simply broadcast its packets of data and the central can receive them without the need to form a connection. The receiver can then examine the payload and receive the data, as long as the data is small enough. In this story, the peripheral is performing the role of a "broadcaster" while the central would be performing the role of an "observer". When working in this mode, do realize that the data is flowing in only one direction … from the publisher to the observer. If you need to send response back, you will need to form a connection. The rate of advertizing can be set to be a period between 20ms and 10.24seconds in steps of 0.625ms. Thus we can transmit our advertizing packets frequently or slowly. See also: • A BLE Advertising Primer

GAP Advertizing data

There is theory and there is practice and studying BLE gives us the opportunity for both. Let us focus on the notion of a BLE peripheral broadcasting advertizing messages. At a high level it will look like: The advertized data payload has a maximum size of 31 bytes. Each payload is composed of one or more data structures where the format of each structure is: Length Advertising Data Type Data ... The Length is 1 byte in size and indicates how many bytes, following the length byte, this record will be. The number of records in a payload is variable but, of course, the total amount of data has to be 31 bytes or less. Either a length of 0 or an ignorable structure type can be used. Here is an example of a real payload that I received: 020105020A000319C1030302E0FF11094D4C452D3135202020202020202020 Now we can parse this as follows: 02 01 05 – advertising type 0x01 02 0a 00 – advertising type 0x0A 03 19 c1 03 – advertising type 0x19 03 02 e0 ff – advertising type 0x02 11 09 4d 4c 45 2d 31 35 20 20 20 20 20 20 20 20 20 – advertising type 09 Here is a simple routine that will step through the structures … while(!finished) { length = *payload; payload++; if (length != 0) { ad_type = *payload; payload += length; ESP_LOGD(tag, "Type: 0x%.2x, length: %d", ad_type, length); } sizeConsumed += 1 + length; if (sizeConsumed >=31 || length == 0) { finished = 1; } } // !finished Now that we know how to see the structures and access them, the obvious question is what do each of the structures "mean". For most, they are architected in the BLE specification. Each structure (after the length) has a single byte that is the "advertising data type". These single byte number codes are described here: https://www.bluetooth.com/specifications/assigned-numbers/generic-access-profile In our ESP32 environment, we have constant definitions for each: • ESP_BLE_AD_TYPE_FLAG (0x01) – The advert contains a byte of flags that are defined as following: ◦ Bit 0 – LE Limited Discoverable Mode ◦ Bit 1 – LE General Discoverable Mode ◦ Bit 2 – BR/EDR is NOT supported. ◦ Bit 3 – Indicates whether LE and BR/EDR Controller operates simultaneously ◦ Bit 4 – Indicates whether LE and BR/EDR Host operates simultaneously ◦ Bits 5-7 – Reserved. • ESP_BLE_AD_TYPE_16SRV_PART (0x02) – Incomplete list of 16bit service class UUIDs. • ESP_BLE_AD_TYPE_16SRV_CMPL (0x03) • ESP_BLE_AD_TYPE_32SRV_PART (0x04) • ESP_BLE_AD_TYPE_32SRV_CMPL (0x05) • ESP_BLE_AD_TYPE_128SRV_PART (0x06) – Incomplete list of 16bit service class UUIDs. • ESP_BLE_AD_TYPE_128SRV_CMPL (x07) • ESP_BLE_AD_TYPE_NAME_SHORT (0x08) – Shortened local name. • ESP_BLE_AD_TYPE_NAME_CMPL (0x09) – Complete local name. • ESP_BLE_AD_TYPE_TX_PWR (0x0A) • ESP_BLE_AD_TYPE_DEV_CLASS (0x0D) • ESP_BLE_AD_TYPE_SM_TK (0x10) • ESP_BLE_AD_TYPE_SM_OOB_FLAG (0x11) • ESP_BLE_AD_TYPE_INT_RANGE (0x12) • ESP_BLE_AD_TYPE_SOL_SRV_UUID (0x14) • ESP_BLE_AD_TYPE_128SOL_SRV_UUID (0x15) • ESP_BLE_AD_TYPE_SERVICE_DATA (0x16) • ESP_BLE_AD_TYPE_PUBLIC_TARGET (0x17) • ESP_BLE_AD_TYPE_RANDOM_TARGET (0x18) • ESP_BLE_AD_TYPE_APPEARANCE (0x19) – It is likely this conforms to the assigned numbers found here https://www.bluetooth.com/specifications/gatt/viewer? attributeXmlFile=org.bluetooth.characteristic.gap.appearance.xml and • ESP_BLE_AD_TYPE_ADV_INT (0x1A) • ESP_BLE_AD_TYPE_32SOL_SRV_UUID (0x1B) • ESP_BLE_AD_TYPE_32SERVICE_DATA (0x1C) • ESP_BLE_AD_TYPE_128SERVICE_DATA (0x1D) • ESP_BLE_AD_MANUFACTURER_SPECIFIC_TYPE (0xFF) – Custom payload. With this in mind, if we look at our sample data … we can see that it means: 02 01 05 – advertising type 0x01 – ESP_BLE_AD_TYPE_FLAG 02 0a 00 – advertising type 0x0A – ESP_BLE_AD_TYPE_TX_PWR 03 19 c1 03 – advertising type 0x19 – ESP_BLE_AD_TYPE_APPEARANCE 03 02 e0 ff – advertising type 0x02 – ESP_BLE_AD_TYPE_16SRV_PART 11 09 4d 4c 45 2d 31 35 20 20 20 20 20 20 20 20 20 – advertising type 0x09 – ESP_BLE_AD_TYPE_NAME_CMPL Now we can consult the specification and determine what the data part of each advertising type means and we have our information. Because examining the advertising data of a GAP message is so important and so common, the ESP-IDF contains a very useful utility function that takes as an input the advertisement data and looks for an advertising type structure within it of a defined type. The function is called esp_ble_resolv_adv_data(). See also: • esp_ble_resolve_adv_data

Advertizability – limited and general

Is "advertizability" even a real word? I think not … but it describes what we want to cover. When we have a bluetooth peripheral, we somehow need to make it available to be discovered by a bluetooth central in order for the central to make requests of the peripheral. The way we achieve this is by making the peripheral visible by it broadcasting its advertising packets. When a peripheral starts advertising, it can tag the advert as either limited-discoverable or general-discoverable. The notion here is that limited-discoverable devices have "recently" been enable to start broadcasting while "general-discoverable" devices usually broadcast continuously. So what you may ask? Well, think of a user looking for a device. It is not uncommon for a new device to be added by pressing some button on it to start it broadcasting so that it can be found and paired. We want this new device to show up higher in the list of found devices than others … and this can be achieved by recognizing that a device which only broadcasts for a limited period of time will enable its "limited-discoverable" flag and provide a hint to the software running on the central that the device is likely going to be the one the user is looking for. Now we have introduced the concept of an advertizing packet having two flags called "limited-discoverable" and "general-discoverable", only one of which may be set at a time. This leaves one more choice which is that neither "limited-discoverable" nor "general-discoverable" are set. This is an allowed transmission and the packet will still be detected by the observer however this indicates that no connection should be attempted and that the peripheral should not be listed to the user.

Filtering devices=

When we are looking for a device, we may find more devices than we want. Rather than present all devices to the user, we can choose to filter the set of available devices and hide the ones that we believe are of no interest. For example, if I am running a heart-rate monitor application on my phone, I'm really only interested in devices that can provide such information. My latest bluetooth headset or the outside temperature is not what I'm looking for. What we want to do is filter the set of ALL found devices down to only the ones that match some criteria of interest to us. In order for that to happen, each device must not only advertize its existence, but also supply information that can be used to include or eliminate it from selection.

Performing a scan

There isn't much value in generating advertising data if no-one is listening. In BLE the act of listening for advertising data is called "scanning". To perform a scan in the ESP32 we perform the following tasks: 1. We register a callback function to handle the received data. 2. We define the parameters for how we would like the scan to be performed. 3. We ask the ESP32 to start scanning. Translating these into ESP32 APIs we have esp_ble_gap_register_callback() to register our callback function to be invoked when a data arrives. We have the esp_ble_gap_set_scan_params() to setup how we wish the scanning to be performed. We have esp_ble_gap_start_scanning() to initiate a scan request. If we need to interrupt our scanning before the specified duration of scanning has completed, we can call esp_ble_gap_stop_scanning(). The function registered when we call esp_ble_gap_register_callback() is where the majority of our function will happen. The parameter to this function is itself a C function which the following signature: void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) The param is a union of structures. Event Type Data Property ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT adv_data_cmpl ESP_GAP_BLE_ADV_DATA_RAW_SET_COMPLETE_EVT adv_data_raw_cmpl ESP_GAP_BLE_ADV_START_COMPLETE_EVT adv_start_cmpl ESP_GAP_BLE_SCAN_PARAM_SET_COMPLETE_EVT scan_param_cmpl ESP_GAP_BLE_SCAN_RESULT_EVT scan_rst ESP_GAP_BLE_SCAN_RSP_DATA_RAW_SET_COMPLETE_EVT scan_rsp_data_raw_cmpl ESP_GAP_BLE_SCAN_RSP_DATA_SET_COMPLETE_EVT scan_rsp_data_cmpl ESP_GAP_BLE_SCAN_START_COMPLETE_EVT scan_start_cmpl The event parameter defines the type of event we have received. Event types include: • ESP_GAP_BLE_ADV_DATA_RAW_SET_COMPLETE_EVT – Raw advertizing data operation complete. • ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT – Called when advertizing data set is complete. Structure parameter is called scan_rsp_data_cmpl. ◦ esp_bt_status_t status – The status of the event. • ESP_GAP_BLE_ADV_START_COMPLETE_EVT – Called when advertizing scan startup is complete. The parameter is a property called scan_start_cmpl which contains: ◦ esp_bt_status_t status – The status of the event. • ESP_GAP_BLE_SCAN_PARAM_SET_COMPLETE_EVT – Called when scan parameters set complete. Structure parameter is called scan_param_cmpl. ◦ esp_bt_status_t status • ESP_GAP_BLE_SCAN_RESULT_EVT – The param is an instance of esp_ble_gap_cb_param_t. Called when one scan result is ready. The structure parameter is called scan_rst. ◦ esp_gap_search_evt_t search_evt – Choices are: ▪ ESP_GAP_SEARCH_INQ_RES_EVT – We have received a search result. ▪ ESP_GAP_SEARCH_INQ_CMPL_EVT – The search is complete. ▪ ESP_GAP_SEARCH_DISC_RES_EVT ▪ ESP_GAP_SEARCH_DISC_BLE_RES_EVT ▪ ESP_GAP_SEARCH_DISC_CMPL_EVT ▪ ESP_GAP_SEARCH_DI_DISC_CMPL_EVT ▪ ESP_GAP_SEARCH_SEARCH_CANCEL_CMPL_EVT ◦ esp_bd_addr_t bda – The address of the device. 6 bytes of data. ◦ esp_bt_dev_type_t dev_type – One of: ▪ ESP_BT_DEVICE_TYPE_BREDR ▪ ESP_BT_DEVICE_TYPE_BLE ▪ ESP_BT_DEVICE_TYPE_DUMO ◦ esp_ble_addr_type_t ble_addr_type – One of ▪ BLE_ADDR_TYPE_PUBLIC ▪ BLE_ADDR_TYPE_RANDOM ▪ BLE_ADDR_TYPE_RPA_PUBLIC ▪ BLE_ADDR_TYPE_RPA_RANDOM ◦ esp_ble_evt_type_t ble_evt_type – One of ◦ int rssi – The signal strength. ◦ uint8_t ble_adv[ESP_BLE_ADV_DATA_LEN_MAX] – The advertized data. ◦ int flag ◦ int num_resps – The number of responses received. This is valid when the search_evt type is ESP_GAP_SEARCH_INQ_CMPL_EVT. • ESP_GAP_BLE_SCAN_RSP_DATA_RAW_SET_COMPLETE_EVT – ???. • ESP_GAP_BLE_SCAN_RSP_DATA_SET_COMPLETE_EVT – Called when the scan response data set is complete. Structure parameter is called: scan_rsp_data_cmpl. ◦ esp_bt_status_t status • ESP_GAP_BLE_SCAN_START_COMPLETE_EVT – ???. A typical series of events received might be: • ESP_GAP_BLE_SCAN_PARAM_SET_COMPLETE_EVT • ESP_GAP_BLE_SCAN_START_COMPLETE_EVT • ESP_GAP_BLE_SCAN_RESULT_EVT – ESP_GAP_SEARCH_INQ_RES_EVT • … • ESP_GAP_BLE_SCAN_RESULT_EVT – ESP_GAP_SEARCH_INQ_CMPL_EVT See also: • esp_ble_gap_register_callback • esp_ble_gap_set_scan_params • esp_ble_gap_start_scanning • esp_ble_gap_stop_scanning

Performing advertising

If our ESP32 is going to be a peripheral, then it will be advertising its existence. The ESP-IDF provides some APIs to achieve that task. At a high level: 1. Call esp_ble_gap_config_adv_data() to specify the content of our periodic advertisement. 2. Call esp_ble_gap_start_advertising() to initiate the periodic advertisement. While superficially simple, we need to consider all the distinct parameters that are available to us … and there are many. We will start with the esp_ble_gap_config_adv_data(). This is where we specify the content of the advertisement payload. An example structure would be: static esp_ble_adv_data_t test_adv_data = { .set_scan_rsp = false, .include_name = true, .include_txpower = true, .min_interval = 0x20, .max_interval = 0x40, .appearance = 0x00, .manufacturer_len = 0, .p_manufacturer_data = NULL, .service_data_len = 0, .p_service_data = NULL, .service_uuid_len = 32, .p_service_uuid = test_service_uuid128, .flag = 0x2, }; Once we have started advertising, we can check the published information using: $ sudo hcitool lescan See also: • esp_ble_gap_config_adv_data • esp_ble_gap_start_advertising • esp_ble_gap_stop_advertising

Bluetooth GATT

The Generic Attribute Protocol (strangely called GATT) provides a mechanism for passing data in a standard form. GATT is always present in BLE. Think of GATT as a way to send and receive data that is "remembered" at the GATT server (while it is running). A client can request the values of data items as well as receive asynchronous notifications as events when something interesting happens at the GATT server. At the high level of the protocol there is the concept of a service. The service is the grouping of functionally related attributes. Within a service are characteristics where a characteristic is a property of that service. Each characteristic type is identified by its own UUID value. In addition a characteristic contains a value, properties, security and may also include descriptors. Within the specification of GATT we have the notion of two roles … namely that of a client and that of a server. A GATT server is commonly a passive entity that waits for requests from GATT clients and services them as they arrive. When a GATT client starts, it assumes little about the GATT server and first performs an inquiry upon the server to determine its characteristics. Every BLE device must include the capability to be a GATT server. The GATT server manages attributes that can be read by or written from a GATT client. How these attributes are stored by the GATT server internally is not defined in the specification and left to the implementers to choose. Within a GATT server, every attribute has a unique "handle" that is 16 bits in length. The value of an attribute is governed by its data type which is defined by a UUID. It has a maximum length of 512 bytes. An attribute also has permissions which govern what one can do against an attribute. The permissions include: • None • Readable • Writable • Both readable and writable This story of raw attributes is further specified in the GATT protocol by the notion of services and characteristics. Think of a service a logical description that a GATT server is prepared to do something that is well defined. If it claims to provide a service, then it must adhere to the contract described by that service. This contract is defined by a set of characteristics and the set of those make up the service. See also: • Generic Attribute Profile (GATT) • generic attributes (gatt) and the generic attribute profile • GATT-Based specifications • GATT services – Assigned numbers • GATT delarations – Assigned number • GATT characteristics – Assigned numbers • gatttool

Being a GATT client

From the ESP32 perspective, to be a GATT client we perform: 1. Register a callback to receive GATT events using esp_ble_gattc_register_callback(). 2. Call esp_ble_gattc_app_register() to register this application. 3. Open a connection to the GATT server using esp_ble_gattc_open(). 4. When an open event arrives, execute a search using esp_ble_gattc_search_service(). When we call esp_ble_gattc_open() we are requesting to open a GATT connection to a specific device. This request is non-blocking and will execute in the background. Eventually we will receive a GATT event indicating the outcome. The event type will be ESP_GATTC_OPEN_EVT. As part of the response data from the ESP_GATTC_OPEN_EVT we will receive a connection identifier (conn_id). This conn_id can be loosely thought of as a socket to the partner device. Once we have formed a connection to the partner device, we can ask it about the services that it offers. We do this by calling esp_ble_gattc_search_service() function. Like other BLE mechanisms, this is an asynchronous operation that results in a series of GATT events being generated. For each service possessed by the device, an ESP_GATTC_SEARCH_RES_EVT is fired and finally, when we have seen all the services, we get an ESP_GATTC_SEARCH_CMPL_EVT. This is illustrated in the following diagram: For each of the services we get back we can start to invoke those services on the device. See also: • esp_ble_gattc_register_callback • esp_ble_gattc_app_register • esp_ble_gattc_open • esp_ble_gattc_search_service • GATT Services

Being a GATT Server

The high level of being a GATT server is: esp_bt_controller_init() esp_bt_controller_enable() esp_bluedroid_init() esp_bluedroid_enable() esp_ble_gatts_register_callback() esp_ble_gap_register_callback() esp_ble_gatts_app_register() The call to esp_ble_gatts_app_register() registers our application. This passes control back to the BLE subsystem and, when ready, will call back the GATT server event handler with an event type of ESP_GATTS_REG_EVT. When we receive that, we then do the next part of the setup: esp_ble_gap_set_device_name() esp_ble_gap_config_adv_data() esp_ble_gatts_create_service() To test that all is working, we can run the hcitool from a Linux system: $ sudo hcitool lescan 24:0A:C4:00:00:96 MYDEVICE See also: • esp_ble_gatts_register_callback • esp_ble_gap_register_callback • esp_ble_gap_set_device_name • esp_ble_gap_config_adv_data • esp_ble_gatts_app_register • esp_ble_gatts_create_service

Service Discovery Protocol

When a client application wishes to request that a connection be established, the client needs to know the port number on which the server is listening. In TCP/IP land, this is achieved by having the client and the server share the implicit knowledge of the port number to use. In Bluetooth, extra functions have been made available. Specifically, there is a service available called the Service Discovery Protocol or "SDP". At the server, when a service is offered, the port number of that service is registered to the local SDP. When a client now wishes to use the target service, it first requests endpoint information from the SDP running on the server. The SDP returns the endpoint information and the client now has all the information it needs to create a direct connection to the desired target service. The unit of information managed by the SDP server is called a "service record" or "SDP record". A command line interface tool called sdptool is available to examine a Bluetooth device's SDP data. A simple command is: $ sdptool browse <Bluetooth Address> This returns a series of records of the form: • Service Name • Service Description • Service Provider • Service RecHandler • Service Class ID List • Protocol Descriptor List • Profile Descriptor List • Language Base Attr List See also: • man(1) – sdptoo l

ESP32 and Bluetooth

Logic appears to be: esp_bt_controller_init() esp_bt_controller_enable(ESP_BT_MODE_BTDM) esp_bluedroid_init() esp_bluedroid_enable() esp_ble_gap_register_callback() esp_ble_gattc_register_callback() esp_ble_gattc_app_register() esp_ble_gap_set_scan_params() esp_ble_gap_start_scanning(20)

Debugging ESP32 Bluetooth

The ESP32 bluetooth implementation is built upon an environment called Bluedroid. The reason this becomes important is that for full understanding of the the ESP32 Bluetooth environment, we need to understand the Bluedroid environment as well. For example, the art of getting lower level diagnostics drops us down to the Bluedroid APIs. For example:

1. include <gatt_api.h> // bluedroid include

… GATT_SetTraceLevel(6); will switch on GATT level tracing.

Bluetooth C Programming in Linux

Within the C / Linux environment we have an implementation of the API stack called "BlueZ". The BlueZ implementation supports RFCOMM, L2CAP, SCO and HCI. In order to perform Bluetooth programming we must install the package called "libbluetooth-dev" using: $ sudo apt-get install libbluetooth-dev When compiling, we need to link with libbluetooth. See also: • An Introduction to Bluetooth Programming

hci_get_route

Retrieve a handle to the specified Bluetooth device or the first available if NULL is supplied. int hci_get_route(bdaddr_t *bdaddr)

hci_open_dev

Open the specified device and get a handle to that device. The returned value is a socket. int hci_open_dev(int dev_id) If the return is -1 then an error was encountered and the details of the error can be found in errno.

hci_inquiry

int hci_inquiry( int dev_id, int len, int max_rsp, const uint8_t *lap, inquiry_info **ii, long flags) where: • dev_id – the device id of the adapter as retruned by hci_get_route. • len – The duration of the scan * 1.28 seconds. • max_rsp – The maximum number of responses we are willing to accept. • lap – may be NULL • ii – Pointer to an array of inquiry_info structures to be populated. The storage must exist and be at least of size max_rsp * sizeof(inquiry_info). • flags ◦ 0 – Cached results allowed ◦ IREQ_CACHE_FLUSH – Any cached values are discarded and only new responses will be used. The inquiry_info is a struct containing: • bdaddr_t bdaddr • uint8_t pscan_rep_mode • uint8_t pscan_period_mode • uint8_t pscan_mode • uint8_t dev_class[3] – The device class is encoded in the assigned numbers – baseband. • uint16_t clock_offset See also: • Assigned numbers – baseband

hci_read_remote_name

Retrieve the display name of a specified device. int hci_read_remote_name( int hci_sock, const baddr_t *addr, int len, char *name, int timeout ) • len – the size of the name buffer to hold the display name. • name – a buffer to hold the display name. • timeout – Maximum number of milliseconds to wait before giving up. On return, a value of 0 indicates success.

str2ba

Convert a string representation of a Bluetooth address into an address. str2ba(const char *str, bdaddr_t *ba) Where str is the string representation of the Bluetooth address and ba is a pointer to a bdaddr_t structure to hold the resulting address.

ba2str

Convert a Bluetooth address to a string. The string buffer must be at least 18 bytes long. ba2str(const bdaddr_t *ba, char *str) The ba is a pointer to a bdaddr_t structure while str is the buffer to be populated with the string representation. sdp_connect() sdp_service_search_attr_req() sdp_record_register()

Bluetooth Audio

Bluetooth speakers and headphones are common so a natural question would be how to get sound out of them from the Pi. The answer is to install an application called PulseAudio. $ sudo apt-get install pulseaudio pulseaudio-module-bluetooth The Bluetooth profile we want to work with is called A2DP (Advanced Audio Distribution Profile).

Bluetooth RFCOMM

A serial protocol is available via Bluetooth and is called "RFCOMM" for "Radio Frequency Communication". When programming with C, we want to create a socket using: int s = socket(AF_BLUETOOTH, SOCK_STREAM, BTPROTO_RFCOMM); A socket address structure for Bluetooth RFCOMM is a struct called sockaddr_rc which contains: • sa_family rc_family – This will always be AF_BLUETOOTH. • bdaddr_t rc_bdaddr – The address of the device to which we wish to connect or listen upon. If any local Bluetooth adapter will suffice when we are a server, we can supply BDADDR_ANY. • uint8_t rc_channel – The channel to which we wish to connect. To be a client of an RFCOMM server, we would use:

1. include <stdio.h>
2. include <unistd.h>
3. include <sys/socket.h>
4. include <bluetooth/bluetooth.h>
5. include <bluetooth/rfcomm.h>

int main(int argc, char *argv[]) { if (argc != 2) { printf("Usage: %s bdaddr\n", argv[0]); return 0; } struct sockaddr_rc addr = {0}; int s, status; char *dest = argv[1]; s = socket(AF_BLUETOOTH, SOCK_STREAM, BTPROTO_RFCOMM); addr.rc_family = AF_BLUETOOTH; addr.rc_channel = 1; str2ba(dest, &addr.rc_bdaddr); status = connect(s,(struct sockaddr *)&addr, sizeof(addr )); if(status == 0) { status = send(s, "hello!", 6, 0); } if(status < 0) { perror("connect"); } close(s); return 0; } while to be a server we would use:

1. include <stdio.h>
2. include <unistd.h>
3. include <sys/socket.h>
4. include <bluetooth/bluetooth.h>
5. include <bluetooth/rfcomm.h>

int main(int argc, char **argv) { struct sockaddr_rc loc_addr = { 0 }, rem_addr = { 0 }; char buf[1024] = { 0 }; int s, client, bytes_read; socklen_t opt = sizeof(rem_addr); // allocate socket s = socket(AF_BLUETOOTH, SOCK_STREAM, BTPROTO_RFCOMM); // bind socket to port 1 of the first available // local bluetooth adapter loc_addr.rc_family = AF_BLUETOOTH; loc_addr.rc_bdaddr = *BDADDR_ANY; loc_addr.rc_channel = (uint8_t) 1; bind(s, (struct sockaddr *)&loc_addr, sizeof(loc_addr)); // put socket into listening mode listen(s, 1); // accept one connection client = accept(s, (struct sockaddr *)&rem_addr, &opt); ba2str(&rem_addr.rc_bdaddr, buf); fprintf(stderr, "accepted connection from %s\n", buf); memset(buf, 0, sizeof(buf)); // read data from the client bytes_read = read(client, buf, sizeof(buf)); if (bytes_read > 0) { printf("received [%s]\n", buf); } // close connection close(client); close(s); return 0; } See also: • RFCOMM with TS 07.10 • man(1) – rfcomm • An Introduction to Bluetooth Programming

Bluetooth tools

l2ping

In TCP/IP networking, we have a tool called ping which sends an ICMP packet over the network to which the partner responds. We can use this command to determine the "liveness" of the partner as well as get the round trip response times. For Bluetooth, we have a similar tool called "l2ping". This command sends a L2CAP echo request to the partner and waits for the response. At the highest level, we use: $ sudo l2ping <bd_addr> where bd_addr is the address of the target device. Here is an example output: $ sudo l2ping 00:1A:7D:DA:71:13 Ping: 00:1A:7D:DA:71:13 from B8:27:EB:62:03:9F (data size 44) ... 44 bytes from 00:1A:7D:DA:71:13 id 0 time 11.32ms 44 bytes from 00:1A:7D:DA:71:13 id 1 time 66.06ms 44 bytes from 00:1A:7D:DA:71:13 id 2 time 19.84ms 44 bytes from 00:1A:7D:DA:71:13 id 3 time 52.38ms As of 05/2016, running the l2ping command on current Raspbian ends after about 5 seconds of pinging with a message: Send failed: Connection reset by peer Current thinking is that this is caused by a deliberate kernel timeout of L2CAP requests that don't result in a connection. The theory believed to be that to save battery life in real Bluetooth devices, if a connection isn't established in the timeout period, don't bother to keep trying and, presumably, waste power resources. The l2ping command is delivered as part of the "bluez" package. See also: • man(1) – l2ping

rfcomm

See also: • man(1) – rfcomm

bluetoothctl

One of the primary tools for working with Bluetooth is called bluetoothctl. Oddly, this command seems to have only the bare bones of a man page (shame). One should run bluetoothctl as root using: $ sudo bluetoothctl If you fail to run it as root, it will just silently sit there until you interrupt it. This command line tool has the following commands: • list – List available controllers • show [ctrl] – Controller information • select <ctrl> – Select default controller • devices – List available devices • paired-devices – List paired devices • power <on/off> – Set controller power • pairable <on/off> – Set controller pairable mode • discoverable <on/off> – Set controller discoverable mode • agent <on/off/capability> – Enable/disable agent with given capability • default-agent – Set agent as the default one • scan <on/off> – Scan for devices • info <dev> – Device information • pair <dev> – Pair with device • trust <dev> – Trust device • untrust <dev> – Untrust device • block <dev> – Block device • unblock <dev> – Unblock device • remove <dev> – Remove device • connect <dev> – Connect device • disconnect <dev> – Disconnect device • version – Display version • quit – Quit program See also:

hciconfig

As mentioned, HCI is the "Host-Controller Interface" which is the layer of communication between the higher level protocols of bluetooth and the bluetooth lower level controller. The "hciconfig" command allows us to execute commands through this logical interface. Running hciconfig by itself will list all the bluetooth devices found on the computer: $ hciconfig hci0: Type: BR/EDR Bus: UART BD Address: B8:27:EB:62:03:9F ACL MTU: 1021:8 SCO MTU: 64:1 UP RUNNING PSCAN RX bytes:19100 acl:150 sco:0 events:457 errors:0 TX bytes:7952 acl:150 sco:0 commands:184 errors:0 Each bluetooth device has a logical identifier of the form "hciX" where the devices are numbered 0, 1, 2 … etc. To refer to a specific device, most of the commands that we issue through hciconfig will take the hciX device identifier to target the correct instance. Some of the more interesting commands we can perform with hciconfig include: • Getting and setting the devices display name property • Enabling/disabling page support • Enabling/disabling scan inquiry support The hciconfig command is supplied as part of the "bluez" package. Some useful commands include: Start LE broadcasting connectable undirected advertising $ sudo hciconfig hci0 leadv 0 Start LE broadcasting non-connectable undirected advertising $ sudo hciconfig hci0 leadv 3 Stop LE broadcasting $ sudo hciconfig hci0 noleadv See also: • hcitool • man(8) – hciconfig

hcidump

This tool is installed through: $ sudo apt-get install bluez-hcidump The tool appears to dump the commands sent through the host/controller interface. Try running with: $ sudo hcidump -x -R to see low-level data.

hcitool

The hcitool is delivered as part of the "bluez" package on Linux. We can issue raw commands through the HCI using: hcitool cmd <OGF> <OCF> Where the combination of the two bytes <OGF> and <OCF> define the command to run. For LE controller commands, the <OGF> is 0x08. To scan for bluetooth LE devices, use: $ sudo hcitool lescan We can start advertising packets. See BT Spec 4.2 Vol 2, Part E 7.8.7 Using: $ sudo hcitool cmd 0x08 0x0008 <Length> <Content> $ hciconfig hci0 leadv 0 See also: • man(1) – hcitoo l • hciconfig

gatttool

Interact with a BLE device at the GATT level. In order to interact with a BLE device at the gatt level, we need its device address. Using "hictool lescan" is a good way to get the address. Typically the program is run with: $ sudo gatttool --device=<Address> --interactive This will return a command prompt that starts with the partner device address: [FF:FF:45:19:14:80][LE]> The sub-commands available to us include: connect [address [address type]] Connect to a remote device disconnect Disconnect from a remote device primary [UUID] Primary Service Discovery included [start hnd [end hnd]] Find Included Services characteristics [start hnd [end hnd [UUID]]] Characteristics Discovery char-desc [start hnd] [end hnd] Characteristics Descriptor Discovery char-read-hnd <handle> Characteristics Value/Descriptor Read by handle char-read-uuid <UUID> [start hnd] [end hnd] Characteristics Value/Descriptor Read by UUID char-write-req <handle> <new value> Characteristic Value Write (Write Request) char-write-cmd <handle> <new value> Characteristic Value Write (No response) sec-level [low | medium | high] Set security level. Default: low mtu <value> Exchange MTU for GATT/ATT Once connected, we can interrogate the device about its primary function by running the "primary" command: [FF:FF:45:19:14:80][LE]> primary attr handle: 0x0001, end grp handle: 0x0005 uuid: 00001800-0000-1000-8000-00805f9b34fb attr handle: 0x0006, end grp handle: 0x0008 uuid: 0000180f-0000-1000-8000-00805f9b34fb attr handle: 0x0009, end grp handle: 0x000b uuid: 00001802-0000-1000-8000-00805f9b34fb attr handle: 0x000c, end grp handle: 0x000e uuid: 0000ffe0-0000-1000-8000-00805f9b34fb Notice specifically the UUIDs. These correspond to the assigned numbers of the GATT specifications. Contrast this with the output of "bluetoothctl info" which shows the following: [bluetooth]# info FF:FF:45:19:14:80 Device FF:FF:45:19:14:80 Alias: FF-FF-45-19-14-80 Appearance: 0x03c1 Icon: input-keyboard Paired: no Trusted: no Blocked: no Connected: yes LegacyPairing: no UUID: Generic Access Profile (00001800-0000-1000-8000-00805f9b34fb) UUID: Immediate Alert (00001802-0000-1000-8000-00805f9b34fb) UUID: Battery Service (0000180f-0000-1000-8000-00805f9b34fb) UUID: Unknown (0000ffe0-0000-1000-8000-00805f9b34fb) For example, 0x...1802… is the Immediate Alert service. Looking back at the gatttool output, for each of the services, we see a "handle range". This describes the handles to the characteristics offered by that service. From the handles, we can ask the device what characteristics these represent: [FF:FF:45:19:14:80][LE]> char-desc 0x0009 0x000b handle: 0x0009, uuid: 00002800-0000-1000-8000-00805f9b34fb handle: 0x000a, uuid: 00002803-0000-1000-8000-00805f9b34fb handle: 0x000b, uuid: 00002a06-0000-1000-8000-00805f9b34fb The UUIDs of the declarations and characteristics can then be examine in the assignednumbers lists. For example • 0x...2800… – GATT Primary Service Declaration • 0x...2803 … – GATT Characteristic Declaration • 0x...2A06... – Alert Level See also: • Bluetooth GATT • man(1) – gatttoo l

Bluetooth examples

The iTag peripheral

The iTag is a cheap little thingy ($2-$4 on eBay) that is a bluetooth device. Its purpose is to form a connection with a bluetooth master. If the connection is subsequently lost, the device starts beeping. In addition, the device can receive a "ping" from the master to instruct it to immediately start beeping. In all these cases, it acts as a useful device to help you locate it should it get lost. Now imagine connecting it to your keys, pet, kid or other loose-able thing and now we have a potential of either being notified that it is out of range or else it will beep to say "help me". This makes a great device for testing the ESP32's ability to work as a BLE master. If we run a BLE scan and listen for advertising packets, we will find that it shows up. From there we can get its bluetooth address. For example, when I ran an ESP32 to look for devices, the ESP32 found my tag with the address "FF:FF:45:14:80". Once the ESP32 found the address, I was the able to issue an open request to it which succeeded. Once I had an open connection, I issued a search request upon it and four services were returned. These were: • UUID: 0x1800 • UUID: 0x1802 • UUID: 0x180f • UUID: 0xffe0 Since these were 16bit UUIDs, that is the indication that they are specified by the bluetooth special interest group (SIG). A search at the GATT services web page found the first three: • UUID: 0x1800 – Generic Access • UUID: 0x1802 – Immediate Alert • UUID: 0x180f – Battery Service Great … this is starting to make sense. Now we can drill down into the characteristics for each service. From the bluetooth specs, the characteristics possible are: • UUID: 0x1800 – Generic Access ◦ UUID: 0x2a00 – Device Name ◦ UUID: 0x2a01 – Appearance ◦ UUID: 0x2a02 – Peripheral Privacy Flag ◦ UUID: 0x2a03 – Reconnection Address ◦ UUID: 0x2a04 – Peripheral Preferred Connection Parameters • UUID: 0x1802 – Immediate Alert ◦ UUID: 0x2a06 – Alert Level • UUID: 0x180f – Battery Service ◦ UUID: 0x2a19 – Battery Level When we actually performed a characteristics query, what was returned were: • UUID: 0x1800 – Generic Access ◦ UUID: 0x2a00 – Device Name ◦ UUID: 0x2a01 – Appearance • UUID: 0x1802 – Immediate Alert ◦ UUID: 0x2a06 – Alert Level • UUID: 0x180f – Battery Service ◦ UUID: 0x2a19 – Battery Level Close enough to what we expected.

Smart Watch / The TW64 Band

The TW64 watch/band can be found on eBay for about $9. Performing an eBay search using "TW64" will turn up many. We start by running a BLE scan: $ sudo hcitool lescan Which came back with: LE Scan ... A4:C1:38:77:1A:19 KeepBand A4:C1:38:77:1A:19 (unknown) And now we know the device address. Of course, each instance will vary. next we can connect to the device and ask it about itself: $ sudo gatttool --device=A4:C1:38:77:1A:19 --interactive > connect > primary attr handle: 0x0001, end grp handle: 0x0007 uuid: 00001800-0000-1000-8000-00805f9b34fb attr handle: 0x0008, end grp handle: 0x000b uuid: 00010203-0405-0607-0809-0a0b0c0d1911 attr handle: 0x000c, end grp handle: 0x0011 uuid: 66880000-0000-1000-8000-008012563489 and similarly, we can also run: $ sudo bluetoothctl

1. info A4:C1:38:77:1A:19

Device A4:C1:38:77:1A:19 Name: KeepBand Alias: KeepBand Appearance: 0x0180 Paired: no Trusted: no Blocked: no Connected: no LegacyPairing: no UUID: Human Interface Device (00001812-0000-1000-8000-00805f9b34fb) UUID: Battery Service (0000180f-0000-1000-8000-00805f9b34fb) Interestingly, notice the distinction in services offered. A search at the GATT services web page found: • UUID: 0x1800 – Generic Access • UUID: 0x180f – Battery Service • UUID: 0x1812 – Human Interface Device Again using gattool, we can ask for the description of characteristics: char-desc handle: 0x0001, uuid: 00002800-0000-1000-8000-00805f9b34fb handle: 0x0002, uuid: 00002803-0000-1000-8000-00805f9b34fb handle: 0x0003, uuid: 00002a00-0000-1000-8000-00805f9b34fb handle: 0x0004, uuid: 00002803-0000-1000-8000-00805f9b34fb handle: 0x0005, uuid: 00002a01-0000-1000-8000-00805f9b34fb handle: 0x0006, uuid: 00002803-0000-1000-8000-00805f9b34fb handle: 0x0007, uuid: 00002a04-0000-1000-8000-00805f9b34fb handle: 0x0008, uuid: 00002800-0000-1000-8000-00805f9b34fb handle: 0x0009, uuid: 00002803-0000-1000-8000-00805f9b34fb handle: 0x000a, uuid: 00010203-0405-0607-0809-0a0b0c0d2b12 handle: 0x000b, uuid: 00002901-0000-1000-8000-00805f9b34fb handle: 0x000c, uuid: 00002800-0000-1000-8000-00805f9b34fb handle: 0x000d, uuid: 00002803-0000-1000-8000-00805f9b34fb handle: 0x000e, uuid: 66880001-0000-1000-8000-008012563489 handle: 0x000f, uuid: 00002803-0000-1000-8000-00805f9b34fb handle: 0x0010, uuid: 66880002-0000-1000-8000-008012563489 handle: 0x0011, uuid: 00002902-0000-1000-8000-00805f9b34fb