What this is about: Export AP logs for successful logins to a txt file. Use the txt file to compare each following login from the same AP: Is new login on the list or not. So, 1) Find and export the relevant data 2) Create a Pipeline and test new logins against this data. I am no docker/Graylog expert. This is more like 'note-to-self" kind of thing. Please check with Docker or Graylog documentation.
I would like to have some ideas about the whereabouts of my APs. Especially logins. Ideally OpenWrt would simply log successful/failed logins and that would be the end of this. However, I cannot figure out how to log the failed attempt ... still. What I can get from OpenWrt is successful logins. Like:
Feb 23 10:31:40 SQUARE dnsmasq-dhcp[5815]: DHCPDISCOVER(br-lan) 192.168.3.214 8a:9e:63:c5:52:30
Feb 23 10:31:40 SQUARE dnsmasq-dhcp[5815]: DHCPOFFER(br-lan) 192.168.3.214 8a:9e:63:c5:52:30
Feb 23 10:31:40 SQUARE dnsmasq-dhcp[5815]: DHCPREQUEST(br-lan) 192.168.3.214 8a:9e:63:c5:52:30
Feb 23 10:31:40 SQUARE dnsmasq-dhcp[5815]: DHCPACK(br-lan) 192.168.3.214 8a:9e:63:c5:52:30 bingtrek
Feb 23 10:31:50 SQUARE dnsmasq-dhcp[5815]: DHCPDISCOVER(br-lan) 192.168.3.214 8a:9e:63:c5:52:30
Feb 23 10:31:50 SQUARE dnsmasq-dhcp[5815]: DHCPOFFER(br-lan) 192.168.3.214 8a:9e:63:c5:52:30
Feb 23 10:31:50 SQUARE dnsmasq-dhcp[5815]: DHCPREQUEST(br-lan) 192.168.3.214 8a:9e:63:c5:52:30
Feb 23 10:31:50 SQUARE dnsmasq-dhcp[5815]: DHCPACK(br-lan) 192.168.3.214 8a:9e:63:c5:52:30 bingtrek
Feb 23 11:46:32 SQUARE dnsmasq-dhcp[5815]: DHCPREQUEST(br-lan) 192.168.3.126 5b:14:f1:c9:d2:7d
Feb 23 11:46:32 SQUARE dnsmasq-dhcp[5815]: DHCPACK(br-lan) 192.168.3.126 5b:14:f1:c9:d2:7d piptrip
Above is all about successful login events (and generated nonsense for this write). So, if I can only get successful logins and I really is most interested in failed or new logins, I will have to 1) create a list off known clients (previous logged in clients) and 2) compare this list against each new DHCPACK. It is not ideal: Hard to imagen a successful login from an unknown device and failed logins are still not logged. Also; the 'all clear' list needs to be updated for each new valid client manually. For the 5 or 6 devices connected to the AP and not much else happening, it is not perfect, but it is something.
Logs from the OpenWrt AP ends up in Graylog and a GROK extractor is configured (format is slightly different):
SQUARE\s%{GREEDYDATA:source}\[%{NUMBER:sourcenr}\]\:\s%{GREEDYDATA:message}\(br-lan\)\s(%{IPV4:client})?(\s)?%{COMMONMAC:mac}(\s)?(%{WORD:hostname})?
From the above GROK patten I get the below fields which are much easier to work with (The last part of the GROK patten was added because some devices are not providing a hostname. Therefore, hostname is optional). Fields generated from the above GROK patten:
BASE10NUM 5815
client 192.168.3.213
facility system daemon
facility_num 3
hostname piptrip
level 6
mac 5b:14:f1:c9:d2:7d
message DHCPACK
source dnsmasq-dhcp
sourcenr 5815
timestamp 2022-08-18 15:41:00.000
It is not much but IP, MAC and hostname is logged for client logins. First column header is "mac". Second column header is "hostname". Exported distinct data from Graylog (https://docs.graylog.org/docs/csv-export):
"mac","hostname"
"8a:9e:63:c5:52:30","bingtrek"
"5b:14:f1:c9:d2:7d","vintagestar"
"5d:2f:6d:9f:2e:df","piptrip"
"d9:ba:c5:6f:21:b8","starstah"
"5a:de:5b:83:fa:a7","purepip"
The above format is generated from Graylog CSV export, but Graylog is very flexible when importing the data so the format should not matter much. With the white-list prepared, it is time to setup the Lookup table and the pipeline to use it.
The Lookup Table consist of 3 parts: Lookup table, Cache and Data Adapters. First create the Data Adapter and Cache and with these two parts, you can create the Lookup Table. The Data Adapter provide the actual data for the lookup table. I.e., the data adapter will point to the generated whitelist txt file. There is a lot of different options here for making the txt file available for the Data Adapter. A Graylog container will already have some persisted storage configured for "/usr/share/graylog/data". Just need to find the mount point. Snip from docker-compose.yml:
...
volumes:
- graylog_data:/usr/share/graylog/data
...
The same Graylog data volume:
$ docker volume ls
DRIVER VOLUME NAME
...
local graylog_data
...
$
Find local mount point for Graylog_data volume:
$ docker volume inspect graylog_data
...
"Mountpoint": "/var/lib/docker/volumes/graylog_graylog_data/_data",
...
$
So, I can copy the lookup mac/hostname txt file to "/var/lib/docker/volumes/greylog_greylog_data/_data" and inside the Graylog container, this file will be available in: "/usr/share/graylog/data". From the host:
sudo ls -l /var/lib/docker/volumes/graylog_graylog_data/_data
total 32
drwxr-x--- 2 1100 1100 4096 Jan 17 2022 config
drwxr-xr-x 2 1100 1100 4096 Jan 17 2022 contentpacks
drwxr-x--- 2 1100 1100 4096 Jan 5 2022 data
drwxr-x--- 3 1100 1100 4096 Aug 19 15:28 journal
drwxr-xr-x 2 1100 1100 4096 Jan 17 2022 libnative
drwxr-x--- 2 1100 1100 4096 Jan 5 2022 log
-rw-r--r-- 1 root root 158 Aug 11 16:15 mac.csv <- mac/hostname txt file.
drwxr-x--- 2 1100 1100 4096 Jan 5 2022 plugin
$
And inside the container:
$ ls -l /usr/share/graylog/data
total 32
drwxr-x--- 2 graylog graylog 4096 Jan 17 2022 config
drwxr-xr-x 2 graylog graylog 4096 Jan 17 2022 contentpacks
drwxr-x--- 2 graylog graylog 4096 Jan 5 2022 data
drwxr-x--- 3 graylog graylog 4096 Aug 19 13:31 journal
drwxr-xr-x 2 graylog graylog 4096 Jan 17 2022 libnative
drwxr-x--- 2 graylog graylog 4096 Jan 5 2022 log
-rw-r--r-- 1 root root 158 Aug 11 14:15 mac.csv <- Point to this.
drwxr-x--- 2 graylog graylog 4096 Jan 5 2022 plugin
$
1) Data Adapter: From Graylog: System > Lookup Tables. Click "Data Adapter" (top right) and "Create data adapter".
| Data Adapter type | CSV file | |
| Title and Description | All square MAC addresses | Anything you want |
| Name | local-mac | Short and descriptive |
| File path | /usr/share/graylog/data/mac.csv | Point to the mac/hostname txt file |
| Check interval | 300 | Static file, so pretty high value |
| Separator | , | Depends on the format of the txt file. For the above file, the default values are OK |
| Quote character | " | Same as above. Use what is relevant format for your txt file. |
| Key column | mac | The lookup value (MAC address) Header of the first column in the txt file. |
| Value column | hostname | Return value of the lookup. The corresponding hostname of the MAC address. Header of the second column in the txt file. |
| Allow case-insensitive lookups | If relevant for your data |
2) Cache: From Graylog: System > Lookup Tables. Click "Caches" (top right) and "Create cache".
| Cache Type | Node-local, in-memory cache | |
| Title and Description | Anything you want | |
| Name | local-mac | |
| Maximum entries | 50 | What fits your hardware/inputs/data throughput |
| Expire after access | disabled | What fits your hardware/inputs/data throughput |
| Expire after write | disabled | What fits your hardware/inputs/data throughput |
3) Lookup table: From Graylog: System > Lookup Tables. Click "Lookup Tables" (top right) and "Create lookup table".
| Title and Description | Anything you want | |
| Name | local-mac | Short and descriptive |
| Enable single default value | Enable | |
| Default single value | not found (string) | The lookup table will return this value then there is no match. |
| Data Adapter | local-mac | Data Adapter created in step 1) |
| Cache | local-mac | Cache created in step 2) |
Test the Lookup Table: System > Lookup Tables. Find the newly configured Lookup table.
Graylog include a very nice test feature for the lookup table. Plug in a MAC addresses from the list in the 'Key' field and the result should be the host name. Likewise plug in anything else and the result should be: ‘not found’
Now we will create a pipeline that will use the Lookup Table to ... lookup new logins from the AP: Test if the new MAC address is on the list. If on the list it will return the corresponding host name. If not on the list, it will return 'not found'.
From Graylog: System > Pipelines. Click "Manage rules" (top right) and "Create Rule" (I assume you already have a pipeline for the relevant stream).
| Description | Something descriptive |
| Rule source | rule "lookup table for allowed mac" when has_field("mac") then let lookupvalue = lookup_value("local-mac", $message.mac); set_field("known_id",lookupvalue); end |
For this to work:
1) Messages in the stream have a field called 'mac' (the field was created with the GROK patten above)
2) The lookup table is called: local-mac (created in step 3)
What this pipeline does:
Compare the incoming message 'mac' field with the mac column in the Lookup Table. If there is a match, a new field called 'know_id' will be added to the message. The value of the field will be the hostname in the hostname (second) column.
Lots of different ways to present the data on the dashboard. A count for 'not found' would fit the purpose fine. For a test run I use a bar chart for showing login counts for different clients.
