New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
Hi all,
I managed to wrote a Prometheus exporter for the Omada app. For those who don't know, Prometheus (https://prometheus.io/) is a software used for event monitoring and alerting.
Basically with the omada-exporter, I've scraped the metrics regarding the access points that you already can find on the Omada web interface.
What's the advantage on that? Well, you can define your own rule of alerting with the Prometheus Alertmanager, for example:
- If an Access Point is down for more than X minutes, send me an email and a Telegram/Slack/SMS message
- If the radio interference on an Access point is too high in the last hour, send me an email
The second big advantage is that you can keep some historical statistics, and build your fancy graphs with Grafana (https://grafana.com/), see screenshot at the bottom.
If do you think that it is interesting and you would like to use it or improve it, I'm going to put the source code on a public repository in the next few days. The exporter can be further improved by adding other metrics regarding the guests, voucher, etc (every stat that you see on the omada controller).
IMHO, I would like to see tplink providing a minimum of support (like sharing their internal API documentation) on this, as with this component you can really add value on top of the whole Omada ecosistem.
Cheers
Screenshots (the graph can be improved of course, this is a first version):
Global status of the site:
Statistics on the Server running omada
Statistic on a single access point
- Copy Link
- Subscribe
- Bookmark
- Report Inappropriate Content
Hi again,
As promised, here: https://gitlab.com/hc-open/omada-exporter you can find the source code of the omada-exporter with some instructions on how to start it.
I would really appreciate any comments/test/collaboration/pull-request (in increasing order of appreciation )
In the next days, I will share also the Grafana Dashboards that I've done, as soon as I understand how to do it. In the meanwhile, if you build your own dashboard, sharing it is appreciated as well, so we can start building interesting tools!!
Cheers
Ab12
- Copy Link
- Report Inappropriate Content
Hi @ab12,
Cool! Can it be adapted to query/poll the OC-200? Or do you have to run the software version of Omada?
-Jonathan
- Copy Link
- Report Inappropriate Content
Hi Jonathan,
As I've decided to query omada through HTTP instead of quering directly the mongo db, the exporter already works with the OC-200.
If you can access the OC-200 via SSH(i don't have one, so I don't know) and it is running some Linux version I think you can install the exporter directly on the controller. Otherwise, you may install it on a server togheter with the rest of the Prometheus stack.
The exporter is simply a Python3 script shipped with Docker.
- Ab12
- Copy Link
- Report Inappropriate Content
@ab12, that's pretty cool! Yes, it would be very helpful if TP-Link would disclose the API of Omada controller.
- Copy Link
- Report Inappropriate Content
Hi again,
As promised, here: https://gitlab.com/hc-open/omada-exporter you can find the source code of the omada-exporter with some instructions on how to start it.
I would really appreciate any comments/test/collaboration/pull-request (in increasing order of appreciation )
In the next days, I will share also the Grafana Dashboards that I've done, as soon as I understand how to do it. In the meanwhile, if you build your own dashboard, sharing it is appreciated as well, so we can start building interesting tools!!
Cheers
Ab12
- Copy Link
- Report Inappropriate Content
@ab12 Have you looked at pulling the stat's directly from the mongoDB. I do this with powershell but never thought to pump it into influx or prometheus.
[Array]$MyArray = @()
$dbName = "tpeap"
$collectionName = "client"
Add-Type -Path "C:\PS-HAB\MongoDrivers\MongoDB.Bson.dll"
Add-Type -Path "C:\PS-HAB\MongoDrivers\MongoDB.Driver.dll"
$db = [MongoDB.Driver.MongoDatabase]::Create("mongodb://localhost:27217/$($dbName)")
$collection = $db[$collectionName]
$results = $collection.findall()
foreach ($Item in $Results)
{
$MacAddress = $Item | select-string -SimpleMatch "Mac=" | where{ $_ -notlike "apMac*" }
$MacAddress = $MacAddress -replace ('mac=', '')
$APName = $Item | Select-String -SimpleMatch "apName"
$APName = $APName -replace ('apName=', '')
$SSID = $Item | Select-String -SimpleMatch "ssid"
$SSID = $SSID -replace ('ssid=', '')
$SignalToNoise = $Item | Select-String -SimpleMatch "snr"
$SignalToNoise = $SignalToNoise -replace ('snr=', '')
$ConnectionSpeed = $Item | select-string -SimpleMatch "rate" | where{ $_ -notlike "clientRateLimitSetting=*" }
$ConnectionSpeed = $ConnectionSpeed -replace ('rate=', '')
$IsActive = $item | Select-String -SimpleMatch "isActive"
$IsActive = $IsActive -replace ('isActive=', '')
$IsActive = $IsActive.ToString()
#IPaddress and hostname go hand in hand.
###################################################################################
$IPAddress = $Item | Select-String -SimpleMatch "ip" | where{ $_ -like "ip=192.*" }
$IPaddress = $IPAddress -replace ('ip=', '')
try
{
$Hostname = (Resolve-DnsName $IPAddress -Server 192.168.1.1 -DnsOnly -ErrorAction Stop).NameHost
$Hostname = $Hostname.Replace('.localdomain', '')
}
Catch
{
$Hostname = "DHCP-$Ipaddress"
}
###################################################################################
$Downloaded = $Item | Select-String -SimpleMatch "download" | where{ $_ -notlike "clientRateLimitSetting=*" }
$Downloaded = $Downloaded -replace ('download=', '')
$Uploaded = $Item | Select-String -SimpleMatch "upload" | where{ $_ -notlike "clientRateLimitSetting=*" }
$Uploaded = $Uploaded -replace ('upload=', '')
$ConnectedFor = $Item | Select-String -SimpleMatch "duration"
$ConnectedFor = $ConnectedFor -replace ('duration=', '')
$RadioID = $Item | Select-String -SimpleMatch "radioID"
$RadioID = $RadioID -replace ('radioId=', '')
if ($RadioID -eq 1) { $RadioID = '5.0GHZ' }
if ($RadioID -eq 0) { $RadioID = '2.4GHZ' }
#Create the PS Object
#------------------------------------------------------------------------------------------
$item = New-Object PSObject
$Item | Add-Member -type NoteProperty -Name 'Types' -Value 'WIRELESS'
$item | Add-Member -type NoteProperty -Name 'Hostname' -Value $Hostname
$item | Add-Member -type NoteProperty -Name 'APName' -Value $APName
$item | Add-Member -type NoteProperty -Name 'IPAddress' -Value $IPAddress
$item | Add-Member -type NoteProperty -Name 'MAC Address' -Value $MacAddress
$item | Add-Member -type NoteProperty -Name 'SSID' -Value $SSID
$item | Add-Member -type NoteProperty -Name 'RadioID' -Value $RadioID
$item | Add-Member -type NoteProperty -Name 'ConnectionSpeed' -Value $ConnectionSpeed
$item | Add-Member -type NoteProperty -Name 'Signal To Noise' -Value $SignalToNoise
$item | Add-Member -type NoteProperty -Name 'Currently Active' -Value $IsActive
$MyArray += $item
#------------------------------------------------------------------------------------------
if ($IsActive -eq 'true' -or $IsActive -eq 'false')
{
#Write-Host "$Hostname,$IPAddress,$APName,$RadioID,$SSID,$ConnectionSpeed,$SignalToNoise,$MacAddress,$isActive"
#UpdateSQLData -DeviceName $Hostname -APName $APName -IPAdress $IPAddress -RadioID $RadioID -SSID $SSID -ConnectionSpeed $ConnectionSpeed -SignalToNoise $SignalToNoise -MacAddress $MacAddress -Downloaded $Downloaded -Uploaded $Uploaded -isActive $isActive
#$MyArray += "$Hostname,$APName,$IPAddress,$SSID,$RadioID"
}
if ($IsActive -eq 'true')
{
#Write-output "$Hostname|$SSID|$Downloaded|$Uploaded" | Out-file C:\PS-HAB\WiFi.log -Append
#$MyArray += "$Hostname,$APName,$IPAddress,$SSID,$RadioID"
}
} #End For Each
$MyArray | Out-GridView
- Copy Link
- Report Inappropriate Content
- Copy Link
- Report Inappropriate Content
Initially I thought to pull the stats directly from MongoDB indeed. But then I decided to use the HTTP Api, mainly because it keeps the exporter less dependant from the actual setup of the controller. If you're using the OC200 Hardware controller for example, you can install the exporter on a different server. With a MongoDB integration, I guess that you would have to install directly the exporter on the OC200 (don't know whether it would be possible, I don't have one).
Besides that, The HTTP api returns me a JSON which is more easier to parse and to work with. Btw, If anyone is interested in how to use the HTTP api, I can post a mini how-to on the subject. I didn't explored all the possible request, but the data format is really clear and self-explanatory.
ab12
- Copy Link
- Report Inappropriate Content
no you cant ssh into the oc200.
if you can ssh into the eaps i dont know why they dont allow the controller.
- Copy Link
- Report Inappropriate Content
Good work, but I'm having a bit of trouble getting it working for an Omada Controller v4.1.5 running on CentOS 7.
I've tried...
- Directly executed with Python 3.8.2 on a Fedora 32 server
- Installed and executed as instructed in a Docker container on a CentOS 7 server
Both return the same error...
INFO:main.omada:Metric Retriever for metric OmadaAPMetrics started
INFO:main:Start prometheus (exposer) http server on port 8000
Exception in thread OmadaAPMetrics:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/python-modules/app/core/omada_metric_retriever.py", line 32, in run
self._get_metrics()
File "/python-modules/app/core/omada_metric_retriever.py", line 106, in _get_metrics
token = self._get_new_auth_token(session)
File "/python-modules/app/core/omada_metric_retriever.py", line 134, in _get_new_auth_token
token = resp.json()['result']['token']
File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
ERROR:main:An error occured on omada metric thread, exiting...
Any idea what I could be missing?
Thank you!
- Copy Link
- Report Inappropriate Content
Hi Joe,
I think that the problem is in a version mismatch of the omada controller. I've developed this exporter on the old omada controller (version 3.2.10). Given that there are many changes in the new version (v4.x.x), probably there is a change in the API interface and structure too.
In production, I'm still using the old version of the controller, however I will soon (next months) switch to the new one, so I'm interested in developing the exporter for the new version. Are you available for helping me? (testing or developing)
regards
- Copy Link
- Report Inappropriate Content
Information
Helpful: 4
Views: 11012
Replies: 13