OS independent compiler mechanics

plugin development

Introduction

This tutorial gives you a detailed introduction to the nuances of multiplatform compilation required if you want to professionally develop Anteater plugins.

Creating your plugin from scratch using a template

In the T2 documentation, you will find a chapter which explains all the nitty-gritty info about programming plugins, but who reads docs today? Right.

To get you started asap, we supplied a skeleton plugin: t2PSkel which compiles and produces output, and serves as a template for your own plugin. It implements all sorts of things what the core supports, so you may delete code samples you do not need in order to remove clutter.

Let’s look at the skeleton under plugins. The alias tran moves to your T2 installation, where you started the setup.sh script and the tranpl alias brings you directly to the plugins folder.

tranpl

ls 

arpDecode   binSink           dhcpDecode  fnameLabel  httpSniffer  jsonSink     modbus      nDPI         ospfDecode    popDecode       pwX           smbDecode   sqliteSink  stunDecode    tcpStates     txtSink
autogen.sh  cdpDecode         dnsDecode   ftpDecode   icmpDecode   ldapDecode   mongoSink   netflowSink  p0f           portClassifier  radiusDecode  smtpDecode  sshDecode   syslogDecode  telnetDecode  voipDetector
basicFlow   connStat          entropy     geoip       igmpDecode   lldpDecode   mqttDecode  nFrstPkts    pcapd         protoStats      regex_pcre    snmpDecode  sslDecode   t2PSkel       tftpDecode    vrrpDecode
basicStats  descriptiveStats  findexer    gtpDecode   ircDecode    macRecorder  mysqlSink   ntpDecode    pktSIATHisto  psqlSink        sctpDecode    socketSink  stpDecode   tcpFlags      tp0f          wavelet
ls t2PSkel/src 

Makefile.am  t2PSkel.c  t2PSkel.h

You may open the t2PSkel.c file with any editor and have a look inside, just to get a feeling for what is coming. That plugin is the mother of all plugins if you use the t2plugin generation script. It takes care of all necessary configurations for you, which I will explain in details below.

The t2PSkel denotes the simplest src plugin structure. The .c file containing the C code. The .h file containing all definitions, constants and includes and the Makefile.am file being used by the autotools to generate the makefiles. In the documentation (Appendix C: Creating a Custom Plugin), all details for plugin development are described. In this tutorial, we will only concentrate on the very basics to get you started.

Plugins are normally loaded by T2 according to a predefined order, which is defined by the so called plugin number.

plugin range    description
------------------------------------
000 - 099       Global
100 - 199       Basic L2/3/4 plugins
200 - 299       Service and routing
300 - 699       L7 protocols
700 - 799       Math and statistics
800 - 899       Classifier and AI
900 - 999       Output

Certain ranges are predefined to cluster plugins, to preserve some order. Nevertheless, you may pick any arbitrary number, but do NOT choose an already existing one. If you do anyway, they are loaded in alphabetic order, and you get confused as a result. The loading order is important for dependencies between plugins, and the processing order. So a plugin with number 103 can deliver results to a plugin with number 200, vice versa would definitely require a time machine. Appendix E (Status) of the documentation details all the existing plugins, their status and their plugin numbers. So you can look it up and see which numbers are already taken or use the t2plugin -l command.

t2plugin -l 

[Global Plugins]

protoStats         001    Overall statistics about protocols


[Basic Plugins]

basicFlow          100    Overall flow information
basicStats         120    Basic statistics
connStat           500    Connection statistics
macRecorder        110    MAC addresses and manufacturers
portClassifier     111    Classification based on port numbers


[Layer 2 Protocol Plugins]

arpDecode          179    Address Resolution Protocol (ARP)
cdpDecode          207    Cisco Discovery Protocol (CDP)
lldpDecode         206    Link Layer Discovery Protocol (LLDP)
stpDecode          203    Spanning Tree Protocol (STP)


[Layer 3/4 Protocol Plugins]

icmpDecode         140    Internet Control Message Protocol (ICMP)
igmpDecode         204    Internet Group Management Protocol (IGMP)
ospfDecode         202    Open Shortest Path First (OSPF)
sctpDecode         135    Stream Control Transmission Protocol (SCTP)
tcpFlags           130    IP and TCP flags
tcpStates          132    TCP connection tracker
vrrpDecode         220    Virtual Router Redundancy Protocol (VRRP)


[Layer 7 Protocol Plugins]

dhcpDecode         250    Dynamic Host Configuration Protocol (DHCP)
dnsDecode          251    Domain Name System (DNS)
ftpDecode          301    File Transfer Protocol (FTP)
httpSniffer        310    HyperText Transfer Protocol (HTTP)
ircDecode          401    Internet Relay Chat (IRC)
ldapDecode         230    Lightweight Directory Access Protocol (LDAP)
modbus             450    Modbus
mqttDecode         452    MQ Telemetry Transport Protocol (MQTT)
ntpDecode          205    Network Time Protocol (NTP)
popDecode          304    Post Office Protocol (POP)
radiusDecode       255    Remote Authentication Dial-In User Service (RADIUS)
smbDecode          335    Server Message Block (SMB)
smtpDecode         303    Simple Mail Transfer Protocol (SMTP)
snmpDecode         386    Simple Network Management Protocol (SNMP)
sshDecode          309    Secure Shell (SSH)
sslDecode          315    SSL/TLS, OpenVPN
stunDecode         257    STUN, TURN, ICE and NAT-PMP
syslogDecode       260    Syslog
telnetDecode       305    Telnet
tftpDecode         300    Trivial File Transfer Protocol (TFTP)
...

Let’s pick 150 because it is higher than the L4 plugins such as tcpFlags, which we will use later as a dependency (Dependencies, described in the Plugin dependencies tutorial are used to access results from other plugins). The t2plugin script helps you to produce a clone from the skeleton plugin where all the names and numbers match your choice.

t2plugin 

Usage:
    t2plugin [OPTION...] [ACTION...]

Actions:
    -c plugin_name    create a new C plugin
    -l                list all available plugins
    -l=type           list all available plugins by type:
                          g: global,
                          b: basic,
                         l2: layer 2,
                         l4: layer 3/4,
                         l7: layer 7,
                          a: application
                          m: math,
                          c: classifier,
                          o: output

Optional arguments:
    --cpp             create a C++ plugin
    -n num            plugin number to use [default: random]

Help and documentation arguments:
    -h, --help        display this help and exit
t2plugin -c tcpWin -n 150 

C plugin 'tcpWin' created

The script reports if the number exists already, in which case we would need to choose a different one.

If you want to integrate your new plugin into the t2_aliases so that you can navigate just by invoking tcpWin then you need to invoke the following command

source ~/tranalyzer2-0.9.1/scripts/t2_aliases

tcpWin

ls 

autogen.sh  CMakeLists.txt  configure.ac  COPYING  default.config  doc  Makefile.am  meson.build  src  t2plconf  tests

Oups, more files since 0.8.8! We will discuss them in the next section. The default.config defines the default configuration in tcpWin.h. The new command: t2conf --reset tcpWin will look into that file to reset your config to default. But this is not an urgent topic here, it is only convenient for the user, if he messed up the config of a plugin.

Now you have a working plugin residing under the plugins directory and it is added into the t2_aliases. I recommend that you remove the lines we do not need for this tutorial for simplicity and better bug detection. Just compare it with the tcpWin.c and tcpWin.h

The inner organs of a plugin: the manual approach

We integrated now additional faster build methods: meson and cmake. If they are not available t2build automatically switches back to the slower autotools. On the downside the developer has now to deal with more files if he wants to change something manually, unless you only want to cover meson. If you rely only on t2Plugin, you do not need to worry about these details. But you do, otherwise you would have skipped this chapter. So let’s carry on.

Let’s start with the Makefile.am file. As outlined above, it contains compiler information. What is important for now is the _la_SOURCES keyword, which defines all files to be compiled and linked. So, if you write additional .c files for your plugin, then you have to add them here. The rest is not important for now it assures access to the core functions and provides support for MAC computers. It will be discussed later.

Here’s a list of who belongs to who:

backend required files
autotools autogen.sh, configure.ac, Makefile.am, src/Makefile.am
cmake autogen.sh, CMakeLists.txt
meson autogen.sh, meson.build

You see cmake and meson need much less files to configure.

autotools: The Makefile contains all compiler related info of the plugin and its dependents

tcpWin

cat src/Makefile.am 

lib_LTLIBRARIES = libtcpWin.la
libtcpWin_la_SOURCES = tcpWin.c

libtcpWin_la_CFLAGS = \
        -I$(top_srcdir)/../../utils \
        -I$(top_srcdir)/../../tranalyzer2/src

if APPLE
libtcpWin_la_CFLAGS += -D_DARWIN_C_SOURCE  # macOS specific flags
else
libtcpWin_la_CFLAGS += -D_GNU_SOURCE
endif

libtcpWin_la_LDFLAGS = -shrext .so  # extension for shared library

For all: The autogen.sh contains information for all build methods, so you need to take care of it anyway. It contains the name of the plugin, its plugin number and where to find the global autogen.sh

cat autogen.sh 

#!/usr/bin/env bash

# Plugin name
PLUGINNAME=tcpWin

# Plugin execution order, as 3-digit decimal
PLUGINORDER=150

# Source the main autogen.sh
. "$(dirname "$0")/../autogen.sh"

cmake: The CMakeLists.txt file contains plugin number, version …

cat CMakeLists.txt 

make_minimum_required(VERSION 3.0.0 FATAL_ERROR)

set(_plugin_name tcpWin)

project(${_plugin_name}
    HOMEPAGE_URL "https://tranalyzer.com"
    VERSION      0.8.11
    LANGUAGES    C
)
...

meson: The meson.build file contains all kind of version info and dependent info as in the Makefile.am

cat meson.build 

project('tcpWin', 'c',
    version: '0.8.11',
    license: 'GPLv2+',
    default_options: [
        'warning_level=2',
        'buildtype=release',
        'c_std=gnu99'
    ],
    #meson_version: '>= 0.45.0',
)

plugin_name = meson.project_name()

os = host_machine.system()

if os == 'darwin'
    add_project_arguments('-D_DARWIN_C_SOURCE', language: 'c')
elif os == 'linux'
    add_project_arguments('-D_GNU_SOURCE', language: 'c')
endif

inc = include_directories(
    join_paths('..', '..', 'utils'),
    join_paths('..', '..', 'tranalyzer2', 'src'),
)

src = [
    join_paths('src', plugin_name + '.c'),
    join_paths('..', '..', 'utils', 'chksum.c'),
]

shared_module(plugin_name,
    sources: src,
    include_directories: inc,
    name_suffix: 'so',
)

You choose which build method you want, or if you want fallback. If you have all, then t2build takes care of everything automatically about the nitty-gritty things on different OS.

Since we have all compiler configs checked, let’s look into the code of our example plugin tcpWin.c in the following tutorial: The basics: your first flow plugin

See also