New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure

This thread has been locked for further replies. You can start a new thread to share your ideas or ask questions.
12

New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure

This thread has been locked for further replies. You can start a new thread to share your ideas or ask questions.
New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-04-30 10:27:46 - last edited 2020-05-06 12:30:07

Hi all,

 

I managed to wrote a Prometheus exporter for the Omada app. For those who don't know, Prometheus (https://prometheus.io/)  is a software used for event monitoring and alerting.

Basically with the omada-exporter, I've scraped the metrics regarding the access points that you already can find on the Omada web interface.

What's the advantage on that? Well, you can define your own rule of alerting with the Prometheus Alertmanager, for example:

  • If an Access Point is down for more than X minutes, send me an email and a Telegram/Slack/SMS message
  • If the radio interference on an Access point is too high in the last hour, send me an email

 

The second big advantage is that you can keep some historical statistics, and build your fancy graphs with Grafana (https://grafana.com/), see screenshot at the bottom.

 

If do you think that it is interesting and you would like to use it or improve it, I'm going to put the source code on a public repository in the next few days. The exporter can be further improved by adding other metrics regarding the guests, voucher, etc (every stat that you see on the omada controller).

IMHO, I would like to see tplink providing a minimum of support (like sharing their internal API documentation) on this, as with this component you can really add value on top of the whole Omada ecosistem.

 

Cheers

 

Screenshots (the graph can be improved of course, this is a first version):

Global status of the site:

 

 

 

Statistics on the Server running omada

 

Statistic on a single access point

  4      
  4      
#1
Options
1 Accepted Solution
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure-Solution
2020-05-04 14:05:13 - last edited 2020-05-06 12:30:07

Hi again,

 

As promised, here: https://gitlab.com/hc-open/omada-exporter you can find the source code of the omada-exporter with some instructions on how to start it.

 

I would really appreciate any comments/test/collaboration/pull-request (in increasing order of appreciation laugh )

 

In the next days, I will share also the Grafana Dashboards that I've done, as soon as I understand how to do it. In the meanwhile, if you build your own dashboard, sharing it is appreciated as well, so we can start building interesting tools!!

 

Cheers

 

Ab12

Recommended Solution
  2  
  2  
#5
Options
13 Reply
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-04-30 12:36:34

Hi @ab12,

 

Cool!  Can it be adapted to query/poll the OC-200?  Or do you have to run the software version of Omada?

 

-Jonathan

  0  
  0  
#2
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-04-30 12:55:21

@JSchnee21 

 

Hi Jonathan,

 

As I've decided to query omada through HTTP instead of quering directly the mongo db, the exporter already works with the OC-200.

If you can access the OC-200 via SSH(i don't have one, so I don't know) and it is running some Linux version I think you can install the exporter directly on the controller. Otherwise, you may install it on a server togheter with the rest of the Prometheus stack.

The exporter is simply a Python3 script shipped with Docker.

 

- Ab12

  0  
  0  
#3
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-04-30 13:04:33

@ab12, that's pretty cool! Yes, it would be very helpful if TP-Link would disclose the API of Omada controller.

༺ 0100 1101 0010 10ཏ1 0010 0110 1010 1110 ༻
  0  
  0  
#4
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure-Solution
2020-05-04 14:05:13 - last edited 2020-05-06 12:30:07

Hi again,

 

As promised, here: https://gitlab.com/hc-open/omada-exporter you can find the source code of the omada-exporter with some instructions on how to start it.

 

I would really appreciate any comments/test/collaboration/pull-request (in increasing order of appreciation laugh )

 

In the next days, I will share also the Grafana Dashboards that I've done, as soon as I understand how to do it. In the meanwhile, if you build your own dashboard, sharing it is appreciated as well, so we can start building interesting tools!!

 

Cheers

 

Ab12

Recommended Solution
  2  
  2  
#5
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-05-08 16:18:43

@ab12 Have you looked at pulling the stat's directly from the mongoDB.  I do this with powershell but never thought to pump it into influx or prometheus.

 

[Array]$MyArray = @()
$dbName = "tpeap"
$collectionName = "client"
Add-Type -Path "C:\PS-HAB\MongoDrivers\MongoDB.Bson.dll"
Add-Type -Path "C:\PS-HAB\MongoDrivers\MongoDB.Driver.dll"
$db = [MongoDB.Driver.MongoDatabase]::Create("mongodb://localhost:27217/$($dbName)")
$collection = $db[$collectionName]


$results = $collection.findall()
foreach ($Item in $Results)
{
    
    $MacAddress = $Item | select-string -SimpleMatch "Mac=" | where{ $_ -notlike "apMac*" }
    $MacAddress = $MacAddress -replace ('mac=', '')
    
    $APName = $Item | Select-String -SimpleMatch "apName"
    $APName = $APName -replace ('apName=', '')
    
    $SSID = $Item | Select-String -SimpleMatch "ssid"
    $SSID = $SSID -replace ('ssid=', '')
    
    $SignalToNoise = $Item | Select-String -SimpleMatch "snr"
    $SignalToNoise = $SignalToNoise -replace ('snr=', '')
    
    $ConnectionSpeed = $Item | select-string -SimpleMatch "rate" | where{ $_ -notlike "clientRateLimitSetting=*" }
    $ConnectionSpeed = $ConnectionSpeed -replace ('rate=', '')
    
    $IsActive = $item | Select-String -SimpleMatch "isActive"
    $IsActive = $IsActive -replace ('isActive=', '')
    $IsActive = $IsActive.ToString()
    
    #IPaddress and hostname go hand in hand.
    ###################################################################################
    $IPAddress = $Item | Select-String -SimpleMatch "ip" | where{ $_ -like "ip=192.*" }
    $IPaddress = $IPAddress -replace ('ip=', '')
    try
    {
        $Hostname = (Resolve-DnsName $IPAddress -Server 192.168.1.1 -DnsOnly -ErrorAction Stop).NameHost
        $Hostname = $Hostname.Replace('.localdomain', '')
    }
    Catch
    {
        $Hostname = "DHCP-$Ipaddress"
    }
    ###################################################################################
    
    $Downloaded = $Item | Select-String -SimpleMatch "download" | where{ $_ -notlike "clientRateLimitSetting=*" }
    $Downloaded = $Downloaded -replace ('download=', '')
    
    $Uploaded = $Item | Select-String -SimpleMatch "upload" | where{ $_ -notlike "clientRateLimitSetting=*" }
    $Uploaded = $Uploaded -replace ('upload=', '')
    
    $ConnectedFor = $Item | Select-String -SimpleMatch "duration"
    $ConnectedFor = $ConnectedFor -replace ('duration=', '')
    
    
    $RadioID = $Item | Select-String -SimpleMatch "radioID"
    $RadioID = $RadioID -replace ('radioId=', '')
    if ($RadioID -eq 1) { $RadioID = '5.0GHZ' }
    if ($RadioID -eq 0) { $RadioID = '2.4GHZ' }
    
    #Create the PS Object
    #------------------------------------------------------------------------------------------
    $item = New-Object PSObject
    $Item | Add-Member -type NoteProperty -Name 'Types' -Value 'WIRELESS'
    $item | Add-Member -type NoteProperty -Name 'Hostname' -Value $Hostname
    $item | Add-Member -type NoteProperty -Name 'APName' -Value $APName
    $item | Add-Member -type NoteProperty -Name 'IPAddress' -Value $IPAddress
    $item | Add-Member -type NoteProperty -Name 'MAC Address' -Value $MacAddress
    $item | Add-Member -type NoteProperty -Name 'SSID' -Value $SSID
    $item | Add-Member -type NoteProperty -Name 'RadioID' -Value $RadioID
    $item | Add-Member -type NoteProperty -Name 'ConnectionSpeed' -Value $ConnectionSpeed
    $item | Add-Member -type NoteProperty -Name 'Signal To Noise' -Value $SignalToNoise
    $item | Add-Member -type NoteProperty -Name 'Currently Active' -Value $IsActive
    
    $MyArray += $item
    #------------------------------------------------------------------------------------------
    
    
    if ($IsActive -eq 'true' -or $IsActive -eq 'false')
    {
        #Write-Host "$Hostname,$IPAddress,$APName,$RadioID,$SSID,$ConnectionSpeed,$SignalToNoise,$MacAddress,$isActive"
        #UpdateSQLData -DeviceName $Hostname -APName $APName -IPAdress $IPAddress -RadioID $RadioID -SSID $SSID -ConnectionSpeed $ConnectionSpeed -SignalToNoise $SignalToNoise -MacAddress $MacAddress -Downloaded $Downloaded -Uploaded $Uploaded -isActive $isActive
        #$MyArray += "$Hostname,$APName,$IPAddress,$SSID,$RadioID"
    }
    
    if ($IsActive -eq 'true')
    {
        #Write-output "$Hostname|$SSID|$Downloaded|$Uploaded" | Out-file C:\PS-HAB\WiFi.log -Append
        #$MyArray += "$Hostname,$APName,$IPAddress,$SSID,$RadioID"
    }
    
} #End For Each


$MyArray | Out-GridView

  0  
  0  
#6
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-05-08 18:38:56

@ab12 

 

Amazing work on this!

  0  
  0  
#7
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-05-08 20:07:46

@DavidFrankKS 

Initially I thought to pull the stats directly from MongoDB indeed. But then I decided to use the HTTP Api, mainly because it keeps the exporter less dependant from the actual setup of the controller. If you're using the OC200 Hardware controller for example, you can install the exporter on a different server. With a MongoDB integration, I guess that you would have to install directly the exporter on the OC200 (don't know whether it would be possible, I don't have one).

 

Besides that, The HTTP api returns me a JSON which is more easier to parse and to work with. Btw, If anyone is interested in how to use the HTTP api, I can post a mini how-to on the subject. I didn't explored all the possible request, but the data format is really clear and self-explanatory.

 

 

ab12

  0  
  0  
#8
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-05-22 10:42:50

no you cant ssh into the oc200.

if you can ssh into the eaps i dont know why they dont allow the controller.

  0  
  0  
#9
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-09-26 00:26:01

@ab12 

 

Good work, but I'm having a bit of trouble getting it working for an Omada Controller v4.1.5 running on CentOS 7.

 

I've tried...

  1. Directly executed with Python 3.8.2 on a Fedora 32 server
  2. Installed and executed as instructed in a Docker container on a CentOS 7 server

 

Both return the same error... 

 

INFO:main.omada:Metric Retriever for metric OmadaAPMetrics started
INFO:main:Start prometheus (exposer) http server on port 8000
Exception in thread OmadaAPMetrics:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/python-modules/app/core/omada_metric_retriever.py", line 32, in run
    self._get_metrics()
  File "/python-modules/app/core/omada_metric_retriever.py", line 106, in _get_metrics
    token = self._get_new_auth_token(session)
  File "/python-modules/app/core/omada_metric_retriever.py", line 134, in _get_new_auth_token
    token = resp.json()['result']['token']
  File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
ERROR:main:An error occured on omada metric thread, exiting...

 

Any idea what I could be missing?

 

Thank you!

  0  
  0  
#10
Options
Re:New Prometheus omada-exporter, or How to build a customizable alerting and monitoring infrastructure
2020-09-28 06:51:37

@Joe-- 

 

Hi Joe,

 

I think that the problem is in a version mismatch of the omada controller. I've developed this exporter on the old omada controller (version 3.2.10). Given that there are many changes in the new version (v4.x.x),  probably there is a change in the API interface and structure too.

 

In production, I'm still using the old version of the controller, however I will soon (next months) switch to the new one, so I'm interested in developing the exporter for the new version. Are you available for helping me? (testing or developing)

 

 

regards

  1  
  1  
#11
Options