The packet forensics mode

packet mode follow stream

Introduction

This tutorial gives you an introduction to the packet mode of T2. It was designed to enable efficient post-processing and as a means of drill down from the flows to the very packet. It uses the same format as the flow files and therefore can be processed by tawk or any other tool of your choice. Each plugin can contribute to the packet mode, same as with flows. Flows and packets are linked by the unique flow index.

Getting started

Create folders for your data and results

If you have not created a separate data and results directory yet, please do it now. This will greatly facilitate your workflow:

mkdir ~/data ~/results

Reset tranalyzer2 and the plugins configuration

If you have followed the other tutorials, you may have modified some of the core and plugins configuration. To ensure your results match those in this tutorial, make sure to reset everything:

t2conf -a --reset

You can also clean all build files:

t2build -a -c

Empty the plugin folder

To ensure we are not left with some unneeded plugins or plugins which were built using different core configuration, it is safer to empty the plugins folder:

t2build -e -y

Are you sure you want to empty the plugin folder '/home/user/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied

Download the PCAP files

The PCAP files used in this tutorial can be downloaded here:

Please save them in your ~/data folder:

wget --no-check-certificate -P ~/data https://tranalyzer.com/download/data/{annoloc2,faf-exercise}.pcap

Build tranalyzer2 and the required plugins

For this tutorial, we will need to build the core (tranalyzer2) and the following plugins:

As you may have modified some of the automatically generated files, it is safer to use the -r and -f options.

t2build -r -f tranalyzer2 basicFlow basicStats ftpDecode tcpStates txtSink

...

BUILDING SUCCESSFUL

Now you are all set for your first packet mode experience.

Activation of packet mode

The packet mode is activated by adding the -s option to the t2 command line. Now, each packet produces a separate line in the packet file.

t2 -s -r ~/data/faf-exercise.pcap -w ~/results

================================================================================
Tranalyzer 0.9.1 (Anteater), Cobra. PID: 15684, SID: 666
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.9.1
    02: basicStats, 0.9.1
    03: tcpStates, 0.9.1
    04: ftpDecode, 0.9.1
    05: txtSink, 0.9.1
[INF] IPv4 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 481577 (481.58 K)
[INF] IPv6 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 41488 (41.49 K)
Processing file: /home/user/data/faf-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Snapshot length: 65535 (65.53 K)
Dump start: 1258544215.037210000 sec (Wed 18 Nov 2009 11:36:55 GMT)
Dump stop : 1258594491.683288000 sec (Thu 19 Nov 2009 01:34:51 GMT)
Total dump duration: 50276.646078000 sec (13h 57m 56s)
Finished processing. Elapsed time: 0.049407769 sec
Finished unloading flow memory. Time: 0.049423329 sec
Percentage completed: 100.00%
Number of processed packets: 5902 (5.90 K)
Number of processed bytes: 4993414 (4.99 M)
Number of raw bytes: 4993414 (4.99 M)
Number of pcap bytes: 5087870 (5.09 M)
Number of IPv4 packets: 5902 (5.90 K) [100.00%]
Number of A packets: 1986 (1.99 K) [33.65%]
Number of B packets: 3916 (3.92 K) [66.35%]
Number of A bytes: 209315 (209.31 K) [4.19%]
Number of B bytes: 4784099 (4.78 M) [95.81%]
<A packet load>: 105.40
<B packet load>: 1221.68 (1.22 K)
--------------------------------------------------------------------------------
basicStats: Flow max(pktload): 1376 (1.38 K)
basicStats: Flow max(b/s), pkts: 8273466 (8.27 Mb/s), 73
basicStats: Biggest L3 flow talker: 143.166.11.10 (US): 3101 (3.10 K) [52.54%] packets
basicStats: Biggest L3 flow talker: 143.166.11.10 (US): 4268858 (4.27 M) [85.49%] bytes
tcpStates: Aggregated tcpStatesAFlags=0x4a
ftpDecode: Aggregated ftpStat=0x0b
ftpDecode: Number of FTP control packets: 22 [0.37%]
ftpDecode: Number of FTP-DATA packets: 4612 (4.61 K) [78.14%]
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 5902 (5.90 K) [100.00%]
Number of TCP bytes: 4993414 (4.99 M) [100.00%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed      flows: 72
Number of processed IPv4 flows: 72 [100.00%]
Number of processed A    flows: 36 [50.00%]
Number of processed B    flows: 36 [50.00%]
Number of request        flows: 36 [50.00%]
Number of reply          flows: 36 [50.00%]
Total   A/B    flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed   packets/flows: 81.97
Number of processed A packets/flows: 55.17
Number of processed B packets/flows: 108.78
Number of processed total packets/s: 0.12
Number of processed A+B   packets/s: 0.12
Number of processed A     packets/s: 0.04
Number of processed   B   packets/s: 0.08
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<Number of processed flows/s>: 0.00
<Bandwidth>: 792 b/s
<Raw bandwidth>: 795 b/s
Max number of flows in memory: 18 [0.01%]
Memory usage: 0.02 GB [0.03%]
Aggregated flowStat=0x0400000000004000
[INF] IPv4 flows

Nice, we observe total flow symmetry (flow asymmetry = 0), so no lonely flows, all IPv4, and we have FTP packets, which means readable content. Look at the biggest talker, maybe it’s the FTP data flow? Let’s find out.

ls ~/results

faf-exercise_flows.txt  faf-exercise_headers.txt  faf-exercise_packets.txt

An additional packets file has been created. Let’s have a look at it!

The pktNo (packet number) and time enable a synchronization between T2 packet mode and Wireshark. Hence, making the cooperation between both tools easier.

Note the pktIAT, pktTrip and flowDuration columns: they reference times separated by flow and direction. The first denotes the packet Inter-Arrival Time (IAT), the next the trip time between A and B packet of a specific flow and the latter the elapsed time since flow A or B started. These times intervals are very useful for signal processing, traffic mining research and in detail troubleshooting. The first bit in flowStat marks the direction of the flow, so you can follow the timing information between different packets and flows

head -28 ~/results/faf-exercise_packets.txt | tcol

%pktNo  flowInd  flowStat            time                  pktIAT       pktTrip      flowDuration  numHdrs  hdrDesc       vlanID  srcMac             dstMac             ethType  srcIP           srcIPCC  srcIPOrg                     srcPort  dstIP           dstIPCC  dstIPOrg                     dstPort  l4Proto  pktLen  udpLen  snapL4Len  snapL7Len  l7Len  tcpStatesAFlags  ftpStat  l7Content
1       1        0x0400000000004000  1258544215.037210000  0.000000000  0.000000000  0.000000000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1258     77.67.44.206    gb       Akamai Technologies          80       6        66      0       28         0          0      0x00             0x00
2       1        0x0400000000004001  1258544215.202900000  0.000000000  0.165690000  0.000000000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1258     6        62      0       28         0          0      0x00             0x00
3       1        0x0400000000004000  1258544215.203358000  0.166148000  0.000458000  0.166148000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1258     77.67.44.206    gb       Akamai Technologies          80       6        64      0       20         0          0      0x00             0x00
4       1        0x0400000000004000  1258544215.203850000  0.000492000  0.000950000  0.166640000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1258     77.67.44.206    gb       Akamai Technologies          80       6        425     0       387        367        367    0x00             0x00     GET /softw/90/update/avg9infoavi.ctf HTTP/1.1\r\nUser-Agent: AVGINET9-WXPPX86 90 AVI=270.14.71/2510 BUILD=707 LOC=1033 LIC=9I-ASXNN-X4WGW-M0XFR-T84VX-3VX02 DIAG=51E OPF=0 PCA=\r\nHost: backup.avg.cz\r\nAccept: */*\r\nAccept-Encoding: identity,deflate,gzip\r\nIf-Modified-Since: Tue, 17 Nov 2009 20:41:10 GMT\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nx-avg-id:78-175947826\r\n\r\n
5       1        0x0400000000004001  1258544215.370055000  0.167155000  0.166205008  0.167155000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1258     6        1434    0       1400       1380       1380   0x00             0x00     HTTP/1.1 200 OK\r\nDate: Wed, 18 Nov 2009 11:39:48 GMT\r\nServer: Apache\r\nLast-Modified: Wed, 18 Nov 2009 09:04:15 GMT\r\nETag: "15c007-cea-478a186e401c0"\r\nAccept-Ranges: bytes\r\nContent-Length: 3306\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nAVG CTF Index File;ver(10)\r\nbin(u7avi1777ff.bin)grp(avi:1777)tm(0911180855)pri(2)len(6637487)\r\nbin(u7avi1777u1323ff.bin)grp(avi:1777)dep(avi:1323)tm(0911180855)pri(2)len(590030)\r\nbin(u7avi1777u1705ff.bin)grp(avi:1777)dep(avi:1705)tm(0911180855)pri(2)len(95323)\r\nbin(u7iavi2511ff.bin)grp(iavi:2511)tm(0911180855)pri(2)len(45641416)\r\nbin(u7iavi2511u2251fo.bin)grp(iavi:2511)dep(iavi:2251)tm(0911180855)pri(2)len(6175342)\r\nbin(u7iavi2511u2353fn.bin)grp(iavi:2511)dep(iavi:2353)tm(0911180855)pri(2)len(4506824)\r\nbin(u7iavi2511u2404fm.bin)grp(iavi:2511)dep(iavi:2404)tm(0911180855)pri(2)len(3342546)\r\nbin(u7iavi2511u2431fl.bin)grp(iavi:2511)dep(iavi:2431)tm(0911180855)pri(2)len(2609643)\r\nbin(u7iavi2511u2458fk.bin)grp(iavi:2511)dep(iavi:2458)tm(0911180855)pri(2)len(1380674)\r\nbin(u7iavi2511u2480fj.bin)grp(iavi:2511)dep(iavi:2480)tm(0911180855)pri(2)len(681743)\r\nbin(u7iavi2511u2490fi.bin)grp(iavi:2511)dep(iavi:2490)tm(0911180855)pri(2)len(555496)\r\nbin(u7iavi2511u2500fh.bin)grp(iavi:2511)dep(iavi:2500)tm(0911180855)pri(2)len(332906)\r\nbin(u7iavi2511u2505fh.bin)grp(iavi:2511)dep(iavi:2505)tm(0911180855)pri(2)len(204942)\r\nbin(u7iavi2511u2508
6       1        0x0400000000004001  1258544215.370067000  0.000012000  0.166217008  0.167167000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1258     6        375     0       341        321        321    0x00             0x00     fg.bin)grp(iavi:2511)dep(iavi:2508)tm(0911180855)pri(2)len(72274)\r\nbin(u7iavi2511u2510ff.bin)grp(iavi:2511)dep(iavi:2510)tm(0911180855)pri(2)len(30540)\r\nbin(u9ichjw4qt.bin)grp(ichjw:4)pri(2)tm(0907131859)len(113488)\r\nbin(u9ichjw4u2ia.bin)grp(ichjw:4)dep(ichjw:2)pri(2)tm(0907131859)len(94470)\r\nbin(u9ichjw4u3gq.bin)grp(ic
7       1        0x0400000000004000  1258544215.370501000  0.166651000  0.000433984  0.333291000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1258     77.67.44.206    gb       Akamai Technologies          80       6        64      0       20         0          0      0x00             0x00
8       1        0x0400000000004001  1258544215.370560000  0.000493000  0.000059008  0.167660000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1258     6        1434    0       1400       1380       1380   0x00             0x00     hjw:4)dep(ichjw:3)pri(2)tm(0907131859)len(13313)\r\nbin(u9idat249b243tc.bin)grp(idat:249)dif(idat:243)pri(2)tm(0911051401)len(621430)\r\nbin(u9idat249b245bp.bin)grp(idat:249)dif(idat:245)pri(2)tm(0911051401)len(197648)\r\nbin(u9idat249b246mj.bin)grp(idat:249)dif(idat:246)pri(2)tm(0911051400)len(159289)\r\nbin(u9idat249b247uh.bin)grp(idat:249)dif(idat:247)pri(2)tm(0911051400)len(95680)\r\nbin(u9idat249le.bin)grp(idat:249)pri(2)tm(0911051400)len(1927074)\r\nbin(u9ifw42jx.bin)grp(ifw:42)pri(2)tm(0911131400)len(541853)\r\nbin(u9ifw42u38we.bin)grp(ifw:42)dep(ifw:38)pri(2)tm(0911131400)len(123575)\r\nbin(u9ifw42u39ke.bin)grp(ifw:42)dep(ifw:39)pri(2)tm(0911131400)len(92745)\r\nbin(u9ifw42u40dc.bin)grp(ifw:42)dep(ifw:40)pri(2)tm(0911131400)len(67292)\r\nbin(u9ifw42u41tz.bin)grp(ifw:42)dep(ifw:41)pri(2)tm(0911131400)len(18393)\r\nbin(x8all234yc.bin)grp(xplph:0012;xplsb:0099;xplsb2:0118;xplsc:0149)tm(0911180700)pri(2)len(1146352)\r\nbin(x8xplph_12gj.bin)grp(xplph:12)tm(0910280700)pri(2)len(4551)\r\nbin(x8xplsb2_118c8.bin)grp(xplsb2:118)tm(0911180700)pri(2)len(4989)\r\nbin(x8xplsb_9946.bin)grp(xplsb:99)tm(0911160700)pri(2)len(985460)\r\nbin(x8xplsb_99d9546.bin)grp(xplsb:99)dif(xplsb:95)tm(0911160700)pri(2)len(32267)\r\nbin(x8xplsb_99d9646.bin)grp(xplsb:99)dif(xplsb:96)tm(0911160700)pri(2)len(3505)\r\nbin(x8xplsb_99d9746.bin)grp(xplsb:99)dif(xplsb:97)tm(0911160700)pri(2)len(644)\r\nbin(x8xplsb_99d9846.bin
9       1        0x0400000000004001  1258544215.370571000  0.000011000  0.000070016  0.167671000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1258     6        520     0       486        466        466    0x00             0x00     )grp(xplsb:99)dif(xplsb:98)tm(0911160700)pri(2)len(581)\r\nbin(x8xplsc_149c8.bin)grp(xplsc:149)tm(0911180700)pri(2)len(151922)\r\nbin(x8xplsc_149d145c8.bin)grp(xplsc:149)dif(xplsc:145)tm(0911180700)pri(2)len(2561)\r\nbin(x8xplsc_149d146c8.bin)grp(xplsc:149)dif(xplsc:146)tm(0911180700)pri(2)len(1921)\r\nbin(x8xplsc_149d147c8.bin)grp(xplsc:149)dif(xplsc:147)tm(0911180700)pri(2)len(1767)\r\nbin(x8xplsc_149d148c8.bin)grp(xplsc:149)dif(xplsc:148)tm(0911180700)pri(2)len(1411)\r\n
10      1        0x0400000000004001  1258544215.370580000  0.000009000  0.000079008  0.167680000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1258     6        54      0       20         0          0      0x00             0x00
11      1        0x0400000000004000  1258544215.370997000  0.000496000  0.000416992  0.333787000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1258     77.67.44.206    gb       Akamai Technologies          80       6        64      0       20         0          0      0x00             0x00
12      1        0x0400000000004000  1258544215.372742000  0.001745000  0.002162016  0.335532000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1258     77.67.44.206    gb       Akamai Technologies          80       6        64      0       20         0          0      0x00             0x00
13      1        0x0400000000004001  1258544215.537951000  0.167371000  0.165208960  0.335051000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1258     6        54      0       20         0          0      0x00             0x00
14      2        0x0400000000004000  1258544216.385370000  0.000000000  0.000000000  0.000000000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1259     77.67.44.206    gb       Akamai Technologies          80       6        66      0       28         0          0      0x00             0x00
15      2        0x0400000000004001  1258544216.551313000  0.000000000  0.165943040  0.000000000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1259     6        62      0       28         0          0      0x00             0x00
16      2        0x0400000000004000  1258544216.551760000  0.166390000  0.000446976  0.166390000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1259     77.67.44.206    gb       Akamai Technologies          80       6        64      0       20         0          0      0x00             0x00
17      2        0x0400000000004000  1258544216.554751000  0.002991000  0.003437952  0.169381000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1259     77.67.44.206    gb       Akamai Technologies          80       6        380     0       342        322        322    0x00             0x00     GET /softw/90/update/u7avi1777u1705ff.bin HTTP/1.1\r\nUser-Agent: AVGINET9-WXPPX86 90 AVI=270.14.71/2510 BUILD=707 LOC=1033 LIC=9I-ASXNN-X4WGW-M0XFR-T84VX-3VX02 DIAG=51E OPF=0 PCA=\r\nHost: backup.avg.cz\r\nAccept: */*\r\nAccept-Encoding: identity,deflate,gzip\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nx-avg-id:78-175947826\r\n\r\n
18      2        0x0400000000004001  1258544216.720958000  0.169645000  0.166207040  0.169645000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1259     6        518     0       484        464        464    0x00             0x00     HTTP/1.1 302 Found\r\nDate: Wed, 18 Nov 2009 11:39:49 GMT\r\nServer: Apache\r\nLocation: http://aa.avg.com/softw/90/update/u7avi1777u1705ff.bin\r\nContent-Length: 238\r\nConnection: close\r\nContent-Type: text/html; charset=iso-8859-1\r\n\r\n<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>302 Found</title>\n</head><body>\n<h1>Found</h1>\n<p>The document has moved <a href="http://aa.avg.com/softw/90/update/u7avi1777u1705ff.bin">here</a>.</p>\n</body></html>\n
19      2        0x0400000000004001  1258544216.720970000  0.000012000  0.166219008  0.169657000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1259     6        54      0       20         0          0      0x00             0x00
20      2        0x0400000000004000  1258544216.721401000  0.166650000  0.000431040  0.336031000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1259     77.67.44.206    gb       Akamai Technologies          80       6        64      0       20         0          0      0x00             0x00
21      2        0x0400000000004000  1258544216.723144000  0.001743000  0.002174016  0.337774000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1259     77.67.44.206    gb       Akamai Technologies          80       6        64      0       20         0          0      0x00             0x00
22      2        0x0400000000004001  1258544216.888595000  0.167625000  0.165451008  0.337282000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   77.67.44.206    gb       Akamai Technologies          80       192.168.1.104   07       Private network              1259     6        54      0       20         0          0      0x00             0x00
23      3        0x0400000000004000  1258544216.908284000  0.000000000  0.000000000  0.000000000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1260     198.189.255.75  us       California State University  80       6        66      0       28         0          0      0x00             0x00
24      3        0x0400000000004001  1258544216.915576000  0.000000000  0.007291968  0.000000000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   198.189.255.75  us       California State University  80       192.168.1.104   07       Private network              1260     6        62      0       28         0          0      0x00             0x00
25      3        0x0400000000004000  1258544216.916026000  0.007742000  0.000449984  0.007742000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1260     198.189.255.75  us       California State University  80       6        64      0       20         0          0      0x00             0x00
26      3        0x0400000000004000  1258544216.929764000  0.013738000  0.014187968  0.021480000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1260     198.189.255.75  us       California State University  80       6        377     0       339        319        319    0x00             0x00     GET /softw/90/update/u7avi1777u1705ff.bin HTTP/1.1\r\nUser-Agent: AVGINET9-WXPPX86 90 AVI=270.14.71/2510 BUILD=707 LOC=1033 LIC=9I-ASXNN-X4WGW-M0XFR-T84VX-3VX02 DIAG=51E OPF=0 PCA=\r\nHost: aa.avg.com\r\nAccept: */*\r\nAccept-Encoding: identity,deflate,gzip\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nx-avg-id:78-175947826\r\n\r\n
27      3        0x0400000000004001  1258544216.936827000  0.021251000  0.007063040  0.021251000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   198.189.255.75  us       California State University  80       192.168.1.104   07       Private network              1260     6        54      0       20         0          0      0x00             0x00

Packet flow tracking

The packet mode provides all features that the flow files contains. Both entries are linked by the flowInd, so you can track each packet back to the flow and vice versa. Hence, extraction of features such as l7Content on a flow basis is a one liner with tawk. Let’s say the flow at index 3 is especially interesting to you:

tawk 'flow(3)' ~/results/faf-exercise_packets.txt | head | tcol

%pktNo  flowInd  flowStat            time                  pktIAT       pktTrip      flowDuration  numHdrs  hdrDesc       vlanID  srcMac             dstMac             ethType  srcIP           srcIPCC  srcIPOrg                     srcPort  dstIP           dstIPCC  dstIPOrg                     dstPort  l4Proto  pktLen  udpLen  snapL4Len  snapL7Len  l7Len  tcpStatesAFlags  ftpStat  l7Content
23      3        0x0400000000004000  1258544216.908284000  0.000000000  0.000000000  0.000000000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1260     198.189.255.75  us       California State University  80       6        66      0       28         0          0      0x00             0x00
24      3        0x0400000000004001  1258544216.915576000  0.000000000  0.007291968  0.000000000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   198.189.255.75  us       California State University  80       192.168.1.104   07       Private network              1260     6        62      0       28         0          0      0x00             0x00
25      3        0x0400000000004000  1258544216.916026000  0.007742000  0.000449984  0.007742000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1260     198.189.255.75  us       California State University  80       6        64      0       20         0          0      0x00             0x00
26      3        0x0400000000004000  1258544216.929764000  0.013738000  0.014187968  0.021480000   3        eth:ipv4:tcp          00:0b:db:4f:6b:10  00:19:e3:e7:5d:23  0x0800   192.168.1.104   07       Private network              1260     198.189.255.75  us       California State University  80       6        377     0       339        319        319    0x00             0x00     GET /softw/90/update/u7avi1777u1705ff.bin HTTP/1.1\r\nUser-Agent: AVGINET9-WXPPX86 90 AVI=270.14.71/2510 BUILD=707 LOC=1033 LIC=9I-ASXNN-X4WGW-M0XFR-T84VX-3VX02 DIAG=51E OPF=0 PCA=\r\nHost: aa.avg.com\r\nAccept: */*\r\nAccept-Encoding: identity,deflate,gzip\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nx-avg-id:78-175947826\r\n\r\n
27      3        0x0400000000004001  1258544216.936827000  0.021251000  0.007063040  0.021251000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   198.189.255.75  us       California State University  80       192.168.1.104   07       Private network              1260     6        54      0       20         0          0      0x00             0x00
28      3        0x0400000000004001  1258544216.937559000  0.000732000  0.007795008  0.021983000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   198.189.255.75  us       California State University  80       192.168.1.104   07       Private network              1260     6        1434    0       1400       1380       1380   0x00             0x00     HTTP/1.1 200 OK\r\nServer: Apache\r\nETag: "0210a9516dd34abc481683f877bd8680:1258533754"\r\nLast-Modified: Wed, 18 Nov 2009 07:55:25 GMT\r\nAccept-Ranges: bytes\r\nContent-Length: 95323\r\nContent-Type: application/octet-stream\r\nDate: Wed, 18 Nov 2009 11:39:50 GMT\r\nConnection: keep-alive\r\n\r\nMZ AVG7 UpdateBin grp(avi:1777)dep(avi:1705)tm(0911180855)pri(2)..7TW;........"....m.b...YbU..&..6.P.B.....jx.\n.n..%....g...8......c....X.c.sO..M............Y.7|..\e...q........w/mb.D#...:.`.H|..(.:\e..wjA/...u....C{.]\e.7.y..8..v....n.5..L.k..U>&te...-.....a..\f`..n. h.....0.......9Ig.s..7^.)..,........ .R..+...f ...xg..xq....;1...F.|....)..*..~.%.I\bo.*......)...P...w.V.q....41....h...w%o..,Ha;.~}..#!.p....{..w\b=A.0...8..IB.;.*...]..w.@..%F[L9(.. ..`..Iq...'......4.&.......\e..Gz0S}`...s.....s...6\).4(..x.J..[do...w./..m.\f[.X.D...z\.. ..F...\nA[....O_...."..te..|b..".......\t..e-..i.q....<&h....SKz.gR.+.<1....n........|..\b-...B..?..".\f./.g.I@..m[s...\eiu3$.t\tL...`...D$..eff..7(.L.\V_..HR!.X.......\f.A#....=...K.[.>..CO.2J...R...k.k.p..ME...\}.v..l_.D\t...D...;c......0~3:A.....\bi\e.7X&..].@.......k?..Qn........,c.`..K.\t.B.M........~\.....>..|._. ...W.YP.....N...u.....s@:..Z.z..n\e."B..Q.M.9..D[.c.z.l...z.G....l..6.yPJ.8.........Q.eE.....oPK.'.s. ..(....+..3........."q...d.....v....@......q\e.+. _YK.`.Zn.c..a..E.q...cI......c..\f.\r0..\n.... ]p..Z=.{./Iz..'..<.d...9...]:...P.}v<...9.h...T9cf../<..U.L
29      3        0x0400000000004001  1258544216.937570000  0.000011000  0.007806016  0.021994000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   198.189.255.75  us       California State University  80       192.168.1.104   07       Private network              1260     6        1434    0       1400       1380       1380   0x00             0x00     .G...*.......SlF...>.(..\......].2`...R?fQY.E\....B.\.]..)Fd>\n.'5...&..^M<..L.4..^U....!\b.......VL.n.%..<J.G.:...bz/L.^.r.........n%Wr'.k\f.g...D...<..f.P............mj...`a.Mc.....B..c.DGz.].e..H.5<f.K...r$....K.-.x..&.....?m....'-.2..0.~.....d/e........4..lx..F\tb.....>...7 Z4..]@,&N.\f.....?I......r.3..a........j.G...\ni.g.\f...d\n..I..k........'..$....6j)svy..u.......T...TH.I..;{Q......\rj.....E..Rc..%.\n...3B.o...)...h].#.<.,1&.......a\f.](..LVKi...z\b..>...Bc.\eY...N.n6l..3..}{~.G.}p .........pPn..c..eQ..m;........O/...+....Z,..$..<.W...\....0RKbHeh'..2.]....E*....a.\ej.7h.9..%Q..R.Z..wP/.JF...3p...[.y..$.h.]..*.%.D.+...#.+.u...>.....I...|.&....-.......%:\t...y....C=.........F....@]X..5&.....W...~Q.%w..d.\b...aZ.....DS..33......Cp._.\t.<......w..!uvt....c....\[Z.Bh'..N...\f..G...Gu..*...\tk..0y....In..:\b*`8......E.. .(R...~..`Z.E-[....;.B..WIR.0....\b^8~....y.6...k..D.V......L7| ..X ...Y...s_......o%Qf2Q0.q.. ...;f5+08..7%.Z......\b.D?.F.]K...@h1D.ah..}Y....#ZF......2.....u]..yc0...<l.E.GO....\e.../g...f.../..+..>..Xw...\f.X....i.q2..W@P.`\7.f.e.X:.-O......nB{o......pu..s.l."Q.....S7D.4k@.Ud..%uxf.."...r.[%...ZZ.....).bS..E.......h.W..0.v.!`.........ix.gh/7Yd.#HO....bo...;....|...F.....e...).x...)....m...A...6!.r..q..Y.W...[.9..H,..4PL;.L...`g.q.-.+.gIk..vy....2...-.....n.O..3.W..p.%.*.wCOm.\t.\f.q.,..[.(V....|....N...K..k.. ..W..jZR...L9...q.z.t.+...<c?.....X....]<...u..'Y.
30      3        0x0400000000004001  1258544216.937579000  0.000009000  0.007815040  0.022003000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   198.189.255.75  us       California State University  80       192.168.1.104   07       Private network              1260     6        1434    0       1400       1380       1380   0x00             0x00     .......R1.cR#..eWk.B....HD...q....p.c..P.t\e....A........\rg..[.x.....>... .jf\n.0..@..[....Z~.a.b..[..5.=\b.7.....\f..^`.8.=.\t..6..\n..eg......p.\n..b....J.(R...O.....G.K3.|.]..]A..\n....z[....K.....\n q\+....S..ox.\t.Hg....i...Q.9s.b4.Y.."o...o..!...p.@.....k.Z.;..I.y.aI.C.......D.G..q..H.h.....L.\....UH.<58..I\b..a.....{.aTy.._...h.8.bQ%.?.....zW\eC..f\C....!x.....O....^{P~'....z9.....8.a...!..{.....Mz....%8...Y/".|...*q=..D.H..@..ZsC...".B...1.MA2..z@......2...S.<]r.....epQ8..Gz.h....V.Qh.....*MYoV..w@...):9.\f.uV.....g'z.,KE:.G$\n.....;../..^(....*.......`.o.....`[...TzF7V..2..o...qU.nE+=n....\na.F..o...\t..h.. .{....}*g....F..,J.9.......ijB...B&..i...A.+.....f ..:ht.;-=.E.....j..2.....h%...\r'...9...\ru._...f...........|I..L..T..../....n.`F.c|.."[g....-...."...v..1y@.....S.]Y....D."..d.-....O:W......~...Y5{.\e..:..."...C..R...%\fnq...~......p....^ZF}n..yB.GFP...-..3..C....~...%r.?`.wT8l.'/M_.6k../.J.1.u._.."W}Z.f.e.".#[.Xh.. .]E....6..X...{..O.0\t.E\......,.._-6r.N.......Zhc......Z.....a...U.....z.*..cW..N8.8........B..h(..51Az..7........^..{.D..........g~EQtM.._....e.;z.?.....~\f..\I.24.>7lQ\fC.X.(D.^x".YJw..0"A....Ix..wR..2..nwt..Qu..?..g.%..\e3.,\r(......A.[Gb.\..4..u38......C\n.e..Y.x.S.)c....z.\f.....e.3..UkY...........U.C]v..*Q..i..\n..-..Q..]\t.<;\f...&.[..0y....0.C....].;....:\f.+.....B..K.\.=...W......6. ...z.....hXd@.h7.7%.. ..E.d[..'..k.s...........jo.O..uaEL......J&.8R..
31      3        0x0400000000004001  1258544216.937598000  0.000019000  0.007834048  0.022022000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:0b:db:4f:6b:10  0x0800   198.189.255.75  us       California State University  80       192.168.1.104   07       Private network              1260     6        1434    0       1400       1380       1380   0x00             0x00     ........%u.He..9......[.x..X...o,.^y.\r*....v...B.)......dN...R#..\....M......C.,.>f.Q[.....7.:.8...-z....^.\e.?..`!..d^..a.!.G.'.6...\n>.o.o...SF..5w.#....h. .J..|..@...k.0...]..A~S#..).. 1..;..F...0.Mf..\b.D.rx..6.+~.%.F...!.m.M...........!.n...~c.........f.....g..6...O.r....-...sC.b.......4..@....R`. ....H..TL..d..P..\n....?)p.(...,..T...C..p..X.m]2....oV`6{w...g.NU.....a.o.......%H...0..h.R.p..g.....fh....[V.L...?@.'-.......?wI....Z)...h.lo!Y.@..e....ab.@l.[Ci\b...Z........h.1...J........m..&.j......b..^....s.K......$.+..\n1.F.....?%..N.......+..Ws.na....L.....U. )..~..(;.c...w\b.v.Q.k\e..3.w3...h.Fu.....i...X..3......u.V....s-."..{.....f..^F......G..l!.\b./.5...C\..Y.......,.9....7.gI....p........].}w.2.......6..m..\e..K.\f...~..q.......TY\t1a.v...".C#3..m...6 ..H.Lb..X..5.b(?..q..........s.\r.IZ.o.\n)\n..3..t/e.....{..../'....Z.B....=.................$6....B.7.p.....0o\t@..m....1.5...t....Z...=.'j!..?:.eXz"q..-.O..1.'c.O-..j.rEA.I...*.bB..]..6Q..\ro..F../.JA.-....$...u...XmS........);K.$.}.."a.}TE.H......n..\b^..]....%.....I~....'.. ..N......!nu..eG....K...../.....Ga...6...V.d.a............*>)...f(^.s<..WR..R.....U......O./..e2....b.b.:.k....c+\rD.......e.V......OkzW..[.....?E..fw".\ta.....!].jQ.t.l.P..W...f.....\t.%..................u..\e.>...l..j../.......cY:@rxp.*-....;.._t..N..-.."......Z&p=ih.2.}\bxV.i.ZGI....V..."...v....=...'K_$0.`a...q;EQS..hn..<'x...n.Ef......,....i.

For researchers, important parameters such as pktLen or l7Len are supplied and can be selected using tawk, and then being piped into further post-processing.

Let’s say that you are only interested in flows that have the FTP flag set:

tawk 'bitsanyset($ftpStat, 0x01)' ~/results/faf-exercise_flows.txt | tcol

%dir  flowInd  flowStat            timeFirst             timeLast              duration       numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  vlanID  srcIP          srcIPCC  srcIPOrg             srcPort  dstIP          dstIPCC  dstIPOrg             dstPort  l4Proto  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT    aveIAT    stdIAT    pktps       bytps     pktAsm  bytAsm      tcpStatesAFlags  ftpStat  ftpCDFindex  ftpCC                          ftpRC                            ftpNumUser  ftpUser      ftpNumPass  ftpPass    ftpNumCP  ftpCP
A     35       0x0400000000004000  1258594162.928342000  1258594185.618346000  22.690004000   1           3        eth:ipv4:tcp  00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800           192.168.1.105  07       "Private network"    49329    143.166.11.10  us       "Dell Technologies"  21       6        11          11           92           1231          0         24        8.363636    10.15629    0       21.78007  2.062728  6.53995   0.484795    4.054649  0       -0.8609222  0x02             0x09     36           USER;PASS;TYPE;PASV;SIZE;RETR                                   1           "anonymous"  1           "IEUser@"  2         "I";"/video/R79733.EXE"
B     35       0x0400000000004001  1258594163.008594000  1258594491.683288000  328.674694000  1           3        eth:ipv4:tcp  00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800           143.166.11.10  us       "Dell Technologies"  21       192.168.1.105  07       "Private network"    49329    6        11          11           1231         92            0         950       111.9091    303.9246    0       306.2558  29.87952  91.89713  0.03346774  3.745345  0       0.8609222   0x42             0x09     36                                          220;331;230;200;227;213;125;226  0                        0                      1         "125 Data connection already open; Transfer startin"

The only real FTP flow is the one with flowInd=35, so let’s select it in the packet file:

tawk 'flow(35)' ~/results/faf-exercise_packets.txt | tcol

%pktNo  flowInd  flowStat            time                  pktIAT         pktTrip        flowDuration   numHdrs  hdrDesc       vlanID  srcMac             dstMac             ethType  srcIP          srcIPCC  srcIPOrg           srcPort  dstIP          dstIPCC  dstIPOrg           dstPort  l4Proto  pktLen  udpLen  snapL4Len  snapL7Len  l7Len  tcpStatesAFlags  ftpStat  l7Content
1266    35       0x0400000000004000  1258594162.928342000  0.000000000    0.000000000    0.000000000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        66      0       28         0          0      0x00             0x01
1267    35       0x0400000000004001  1258594163.008594000  0.000000000    0.080251968    0.000000000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        62      0       28         0          0      0x00             0x01
1268    35       0x0400000000004000  1258594163.009292000  0.080950000    0.000698000    0.080950000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        64      0       20         0          0      0x00             0x01
1269    35       0x0400000000004001  1258594163.087792000  0.079198000    0.078500000    0.079198000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        81      0       47         27         27     0x00             0x01     220 Microsoft FTP Service\r\n
1270    35       0x0400000000004000  1258594163.088491000  0.079199000    0.000699000    0.160149000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        74      0       36         16         16     0x00             0x01     USER anonymous\r\n
1271    35       0x0400000000004001  1258594163.166256000  0.078464000    0.077765000    0.157662000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        126     0       92         72         72     0x00             0x01     331 Anonymous access allowed, send identity (e-mail name) as password.\r\n
1272    35       0x0400000000004000  1258594163.168693000  0.080202000    0.002436992    0.240351000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        72      0       34         14         14     0x00             0x01     PASS IEUser@\r\n
1273    35       0x0400000000004001  1258594163.247178000  0.080922000    0.078485008    0.238584000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        1004    0       970        950        950    0x00             0x01     230-Welcome to the Dell FTP site. A service of Dell Inc., Round Rock, Texas.\r\n    For information about DELL, call +1 800 999 3355 All transfers are logged with\r\n    your host name and email address. If you don't like this policy please disconnect now.\r\n    Please be advised that use constitutes consent to monitoring (Elec Comm Priv Act,\r\n    18 USC 2701-2711). Please see the file readme.txt for disclaimers pertaining to this\r\n    service. If your FTP client crashes or hangs shortly after login, try using a dash\r\n    (-) as the first character of your password. This will turn off the informational\r\n    messages which may be confusing your ftp client.\r\n    ********IN CASE OF PROBLEMS*************************\r\n    ** File Content: send EMAIL to dellbbs@dell.com   **\r\n    ** FTP Server: send EMAIL to hostmaster@dell.com  **\r\n    ** WWW Server: send EMAIL to webmaster@dell.com   **\r\n    ****************************************************\r\n
1274    35       0x0400000000004001  1258594163.247187000  0.000009000    0.078494016    0.238593000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        75      0       41         21         21     0x00             0x01     230 User logged in.\r\n
1275    35       0x0400000000004000  1258594163.247637000  0.078944000    0.000449984    0.319295000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        64      0       20         0          0      0x00             0x01
1276    35       0x0400000000004000  1258594163.249385000  0.001748000    0.002197984    0.321043000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        66      0       28         8          8      0x00             0x01     TYPE I\r\n
1277    35       0x0400000000004001  1258594163.327121000  0.079934000    0.077736000    0.318527000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        74      0       40         20         20     0x00             0x01     200 Type set to I.\r\n
1278    35       0x0400000000004000  1258594163.327845000  0.078460000    0.000724000    0.399503000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        64      0       26         6          6      0x00             0x01     PASV\r\n
1279    35       0x0400000000004001  1258594163.407582000  0.080461000    0.079737024    0.398988000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        104     0       70         50         50     0x00             0x09     227 Entering Passive Mode (143,166,11,10,251,78)\r\n
1283    35       0x0400000000004000  1258594163.487490000  0.159645000    0.079907968    0.559148000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        82      0       44         24         24     0x00             0x09     SIZE /video/R79733.EXE\r\n
1284    35       0x0400000000004001  1258594163.565990000  0.158408000    0.078500032    0.557396000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        67      0       33         13         13     0x00             0x09     213 4255056\r\n
1285    35       0x0400000000004000  1258594163.566694000  0.079204000    0.000704000    0.638352000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        82      0       44         24         24     0x00             0x09     RETR /video/R79733.EXE\r\n
1286    35       0x0400000000004001  1258594163.644188000  0.078198000    0.077494016    0.635594000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        108     0       74         54         54     0x00             0x09     125 Data connection already open; Transfer starting.\r\n
1303    35       0x0400000000004000  1258594163.838277000  0.271583000    0.194088960    0.909935000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        64      0       20         0          0      0x00             0x09
5898    35       0x0400000000004001  1258594185.427515000  21.783327000   21.589238016   22.418921000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        78      0       44         24         24     0x00             0x09     226 Transfer complete.\r\n
5900    35       0x0400000000004000  1258594185.618346000  21.780069000   0.190830976    22.690004000   3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        64      0       20         0          0      0x00             0x09
5902    35       0x0400000000004001  1258594491.683288000  306.255773000  306.064942016  328.674694000  3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        54      0       20         0          0      0x42             0x09

Absolute/relative seq/ack numbers

If the tcpFlags plugin is added, sequence/acknowledgment numbers, window size and certain options are displayed in packet mode. The default of absolute seq/ack numbers can be changed to relative which is helpful when analyzing the evolution of such numbers. In order to change to relative representation move to the tcpFlags directory, open tcpFlags.h and set the constant to relative.

tcpFlags

vi src/tcpFlags.h

...
/* ========================================================================== */
/* ------------------------ USER CONFIGURATION FLAGS ------------------------ */
/* ========================================================================== */

...
#define SEQ_ACK_NUM      1 // 1: SEQ/ACK number feature analysis
...

// The following options require SEQ_ACK_NUM = 1

#define SPKTMD_SEQACKREL 0 // SEQ/ACK numbers representation (-s option):
                           //   0: absolute,
                           //   1: relative

#define SPKTMD_SEQACKHEX 0 // SEQ/ACK numbers representation (-s option):
                           //   0: uint32_t
                           //   1: hex32

...

Note that SPKTMD_SEQACKHEX=0 means that seq/ack numbers are represented as integer. The other option is hex, whatever is suited best for your post-processing. We stick here with the default.

Now recompile the tcpFlags plugin and rerun t2:

t2conf tcpFlags -D SPKTMD_SEQACKREL=1 && t2build tcpFlags

t2 -r ~/data/faf-exercise.pcap -w ~/results/ -s

Extract flow 35 again from ~/results/faf-exercise_packets.txt and look for the seq and ack columns. They are now all relative to the beginning of the flow.

tawk 'flow(35)' ~/results/faf-exercise_packets.txt | tcol

%pktNo  flowInd  flowStat            time               pktIAT      pktTrip     flowDuration  numHdrs  hdrDesc       ethVlanID  srcMac             dstMac             ethType  srcIP          srcIPCC  srcIPOrg         srcPort  dstIP          dstIPCC  dstIPOrg         dstPort  l4Proto  pktLen  l7Len  ipToS  ipID   ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq   ack   seqMax  seqDiff  ackDiff  seqLen  ackLen  seqFlowLen  ackFlowLen  tcpMLen  tcpBFlgt  tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpWS  tcpMSS  tcpTmS  tcpTmER  tcpOptLen  tcpOpts                                  tcpStatesAFlags  ftpStat  l7Content
1266    35       0x0400000000004000  1258594162.928342  0.000000    0.000000    0.000000      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        66      0      0x00   16230  0         0x4000  128    0x5ea0       0x5ea0       0x7ccd       0x7ccd       0x0040   0                       0                     0                 0     0     0       0        0        0       0       0           0           0        0          0x0010    0x0002    0x0000      8192    0      1460    0       0        8          0x02;0x04;0x05;0xb4;0x01;0x01;0x04;0x02  0x00             0x01
1267    35       0x0400000000004001  1258594163.008594  0.000000    0.080252    0.000000      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        62      0      0x00   55468  0         0x4000  239    0x5659       0x5659       0x1d37       0x1d37       0x0040   0                       0                     0                 0     1     0       0        0        0       0       0           0           0        0          0x0010    0x0212    0x0000      4140    0      1380    0       0        8          0x02;0x04;0x05;0x64;0x04;0x02;0x00;0x00  0x00             0x01
1268    35       0x0400000000004000  1258594163.009292  0.080950    0.000698    0.080950      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        64      0      0x00   16231  1         0x4000  128    0x5ea7       0x5ea7       0x5b79       0x5b79       0x0040   0                       0                     0                 1     1     1       1        0        0       0       0           0           0        0          0x0811    0x0010    0x0000      64860   0      1460    0       0        0                                                   0x00             0x01
1269    35       0x0400000000004001  1258594163.087792  0.079198    0.078500    0.079198      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        81      27     0x00   58625  3157      0x4000  239    0x49f1       0x49f1       0xad9d       0xad9d       0x0040   0                       0                     0                 1     1     1       1        0        0       0       0           0           27       27         0x0011    0x0018    0x0000      4140    0      1380    0       0        0                                                   0x00             0x01     220 Microsoft FTP Service\r\n
1270    35       0x0400000000004000  1258594163.088491  0.079199    0.000699    0.160149      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        74      16     0x00   16243  12        0x4000  128    0x5e8b       0x5e8b       0xd384       0xd384       0x0040   0                       0                     0                 1     28    1       0        27       0       27      0           27          16       16         0x0011    0x0018    0x0000      64833   0      1460    0       0        0                                                   0x00             0x01     USER anonymous\r\n
1271    35       0x0400000000004001  1258594163.166256  0.078464    0.077765    0.157662      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        126     72     0x00   61580  2955      0x4000  239    0x3e39       0x3e39       0xf987       0xf987       0x0040   0                       0                     0                 28    17    28      27       16       27      16      27          16          99       72         0x0811    0x0018    0x0000      4156    0      1380    0       0        0                                                   0x00             0x01     331 Anonymous access allowed, send identity (e-mail name) as password.\r\n
1272    35       0x0400000000004000  1258594163.168693  0.080202    0.002437    0.240351      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        72      14     0x00   16244  1         0x4000  128    0x5e8c       0x5e8c       0x5f70       0x5f70       0x0040   0                       0                     0                 17    100   17      16       72       16      72      16          99          30       14         0x0011    0x0018    0x0000      64761   0      1460    0       0        0                                                   0x00             0x01     PASS IEUser@\r\n
1273    35       0x0400000000004001  1258594163.247178  0.080922    0.078485    0.238584      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        1004    950    0x00   64425  2845      0x4000  239    0x2fae       0x2fae       0x41de       0x41de       0x0040   0                       0                     0                 100   31    100     72       14       72      14      99          30          1049     950        0x0811    0x0018    0x0000      4170    0      1380    0       0        0                                                   0x00             0x01     230-Welcome to the Dell FTP site. A service of Dell Inc., Round Rock, Texas.\r\n    For information about DELL, call +1 800 999 3355 All transfers are logged with\r\n    your host name and email address. If you don't like this policy please disconnect now.\r\n    Please be advised that use constitutes consent to monitoring (Elec Comm Priv Act,\r\n    18 USC 2701-2711). Please see the file readme.txt for disclaimers pertaining to this\r\n    service. If your FTP client crashes or hangs shortly after login, try using a dash\r\n    (-) as the first character of your password. This will turn off the informational\r\n    messages which may be confusing your ftp client.\r\n    ********IN CASE OF PROBLEMS*************************\r\n    ** File Content: send EMAIL to dellbbs@dell.com   **\r\n    ** FTP Server: send EMAIL to hostmaster@dell.com  **\r\n    ** WWW Server: send EMAIL to webmaster@dell.com   **\r\n    ****************************************************\r\n
1274    35       0x0400000000004001  1258594163.247187  0.000009    0.078494    0.238593      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        75      21     0x00   64426  1         0x4000  239    0x334e       0x334e       0x2a2a       0x2a2a       0x0040   0                       0                     0                 1050  31    1050    950      0        950     0       1049        30          1070     971        0x0811    0x0018    0x0000      4170    0      1380    0       0        0                                                   0x00             0x01     230 User logged in.\r\n
1275    35       0x0400000000004000  1258594163.247637  0.078944    0.000450    0.319295      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        64      0      0x00   16253  9         0x4000  128    0x5e91       0x5e91       0x5b5b       0x5b5b       0x0040   0                       0                     0                 31    1071  31      14       971      14      971     30          1070        30       0          0x0011    0x0010    0x0000      63790   0      1460    0       0        0                                                   0x00             0x01
1276    35       0x0400000000004000  1258594163.249385  0.001748    0.002198    0.321043      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        66      8      0x00   16254  1         0x4000  128    0x5e88       0x5e88       0x8959       0x8959       0x0040   0                       0                     0                 31    1071  31      0        0        0       0       30          1070        38       8          0x0811    0x0018    0x0000      63790   0      1460    0       0        0                                                   0x00             0x01     TYPE I\r\n
1277    35       0x0400000000004001  1258594163.327121  0.079934    0.077736    0.318527      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        74      20     0x00   1622   -62804    0x4000  239    0x28a4       0x28a4       0xb130       0xb130       0x0044   0                       0                     0                 1071  39    1071    21       8        21      8       1070        38          1090     20         0x0811    0x0018    0x0000      4178    0      1380    0       0        0                                                   0x00             0x01     200 Type set to I.\r\n
1278    35       0x0400000000004000  1258594163.327845  0.078460    0.000724    0.399503      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        64      6      0x00   16255  1         0x4000  128    0x5e89       0x5e89       0xaaa3       0xaaa3       0x0040   0                       0                     0                 39    1091  39      8        20       8       20      38          1090        44       6          0x0011    0x0018    0x0000      63770   0      1460    0       0        0                                                   0x00             0x01     PASV\r\n
1279    35       0x0400000000004001  1258594163.407582  0.080461    0.079737    0.398988      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        104     50     0x00   5259   3637      0x4000  239    0x1a51       0x1a51       0xbf53       0xbf53       0x0040   0                       0                     0                 1091  45    1091    20       6        20      6       1090        44          1140     50         0x0811    0x0018    0x0000      4184    0      1380    0       0        0                                                   0x00             0x09     227 Entering Passive Mode (143,166,11,10,251,78)\r\n
1283    35       0x0400000000004000  1258594163.487490  0.159645    0.079908    0.559148      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        82      24     0x00   16267  12        0x4000  128    0x5e6b       0x5e6b       0xf13a       0xf13a       0x0040   0                       0                     0                 45    1141  45      6        50       6       50      44          1140        68       24         0x0011    0x0018    0x0000      63720   0      1460    0       0        0                                                   0x00             0x09     SIZE /video/R79733.EXE\r\n
1284    35       0x0400000000004001  1258594163.565990  0.158408    0.078500    0.557396      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        67      13     0x00   11024  5765      0x4000  239    0x03f1       0x03f1       0x049e       0x049e       0x0040   0                       0                     0                 1141  69    1141    50       24       50      24      1140        68          1153     13         0x0811    0x0018    0x0000      4208    0      1380    0       0        0                                                   0x00             0x09     213 4255056\r\n
1285    35       0x0400000000004000  1258594163.566694  0.079204    0.000704    0.638352      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        82      24     0x00   16268  1         0x4000  128    0x5e6a       0x5e6a       0xf819       0xf819       0x0040   0                       0                     0                 69    1154  69      24       13       24      13      68          1153        92       24         0x0011    0x0018    0x0000      63707   0      1460    0       0        0                                                   0x00             0x09     RETR /video/R79733.EXE\r\n
1286    35       0x0400000000004001  1258594163.644188  0.078198    0.077494    0.635594      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        108     54     0x00   14255  3231      0x4000  239    0xf728       0xf728       0xb8e3       0xb8e3       0x0040   0                       0                     0                 1154  93    1154    13       24       13      24      1153        92          1207     54         0x0811    0x0018    0x0000      4232    0      1380    0       0        0                                                   0x00             0x49     125 Data connection already open; Transfer starting.\r\n
1303    35       0x0400000000004000  1258594163.838277  0.271583    0.194089    0.909935      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        64      0      0x00   16289  21        0x4000  128    0x5e6d       0x5e6d       0x5b1d       0x5b1d       0x0040   0                       0                     0                 93    1208  93      24       54       24      54      92          1207        92       0          0x0011    0x0010    0x0000      63653   0      1460    0       0        0                                                   0x00             0x09
5898    35       0x0400000000004001  1258594185.427515  21.783327   21.589237   22.418921     3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        78      24     0x00   40815  26560     0x4000  239    0x8f86       0x8f86       0x7425       0x7425       0x0040   0                       0                     0                 1208  93    1208    54       0        54      0       1207        92          1231     24         0x0811    0x0018    0x0000      4232    0      1380    0       0        0                                                   0x00             0x49     226 Transfer complete.\r\n
5900    35       0x0400000000004000  1258594185.618346  21.780069   0.190831    22.690004     3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        64      0      0x00   18617  2328      0x4000  128    0x5555       0x5555       0x5b1d       0x5b1d       0x0040   0                       0                     0                 93    1232  93      0        24       0       24      92          1231        92       0          0x0011    0x0010    0x0000      63629   0      1460    0       0        0                                                   0x00             0x09
5902    35       0x0400000000004001  1258594491.683288  306.255768  306.064941  328.674683    3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        54      0      0x00   49361  8546      0x4000  239    0x6e3c       0x6e3c       0x431f       0x431f       0x0040   0                       0                     0                 1232  93    1232    24       0        24      0       1231        92          1231     0          0x0811    0x0414    0x0000      4232    0      1380    0       0        0

Adding more plugins

Let’s add some more plugins which contribute to the packet file.

t2build icmpDecode macRecorder portClassifier

...
BUILD SUCCESSFUL
t2 -s -r ~/data/faf-exercise.pcap -w ~/results

================================================================================
Tranalyzer 0.9.1 (Anteater), Cobra. PID: 16591, SID: 666
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.9.1
    02: macRecorder, 0.9.1
    03: portClassifier, 0.9.1
    04: basicStats, 0.9.1
    05: tcpFlags, 0.9.1
    06: tcpStates, 0.9.1
    07: icmpDecode, 0.9.1
    08: ftpDecode, 0.9.1
    09: txtSink, 0.9.1
[INF] IPv4 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 481577 (481.58 K)
[INF] IPv6 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 41488 (41.49 K)
[INF] macRecorder: 84110 (84.11 K) short org name records loaded
Processing file: /home/user/data/faf-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Snapshot length: 65535 (65.53 K)
Dump start: 1258544215.037210000 sec (Wed 18 Nov 2009 11:36:55 GMT)
Dump stop : 1258594491.683288000 sec (Thu 19 Nov 2009 01:34:51 GMT)
Total dump duration: 50276.646078000 sec (13h 57m 56s)
Finished processing. Elapsed time: 0.061677920 sec
Finished unloading flow memory. Time: 0.061710031 sec
Percentage completed: 100.00%
Number of processed packets: 5902 (5.90 K)
Number of processed bytes: 4993414 (4.99 M)
Number of raw bytes: 4993414 (4.99 M)
Number of pcap bytes: 5087870 (5.09 M)
Number of IPv4 packets: 5902 (5.90 K) [100.00%]
Number of A packets: 1986 (1.99 K) [33.65%]
Number of B packets: 3916 (3.92 K) [66.35%]
Number of A bytes: 209315 (209.31 K) [4.19%]
Number of B bytes: 4784099 (4.78 M) [95.81%]
<A packet load>: 105.40
<B packet load>: 1221.68 (1.22 K)
--------------------------------------------------------------------------------
basicStats: Flow max(pktload): 1376 (1.38 K)
basicStats: Flow max(b/s), pkts: 8273466 (8.27 Mb/s), 73
basicStats: Biggest L3 flow talker: 143.166.11.10 (US): 3101 (3.10 K) [52.54%] packets
basicStats: Biggest L3 flow talker: 143.166.11.10 (US): 4268858 (4.27 M) [85.49%] bytes
tcpFlags: Aggregated ipFlags=0x0044
tcpFlags: Aggregated tcpFStat=0x4ff1
tcpFlags: Aggregated tcpFlags=0x071f
tcpFlags: Aggregated tcpAnomaly=0x02cc
tcpFlags: Number of TCP SYN retries, seq retries: 0, 27
tcpFlags: Number WinSz below 1: 3 [0.05%]
tcpStates: Aggregated tcpStatesAFlags=0x4a
ftpDecode: Aggregated ftpStat=0x0b
ftpDecode: Number of FTP control packets: 22 [0.37%]
ftpDecode: Number of FTP-DATA packets: 4612 (4.61 K) [78.14%]
--------------------------------------------------------------------------------
...

By invoking the same tawk query as before, we find from portClassifier a human readable output of the port based assignment of the embedded protocol; Here FTP.

macRecorder tells us that there is only one interface pair involved as macPairs is 1. If load balancing is involved or an interface card is broken, there can be more macPairs per flow. Moreover, the manufacturer is decoded from the first three octets of the MAC address.

icmpDecode output will be discussed below.

tcpFlags provides all aggregated information of IP and Layer 4. See the IP/TCP troubleshooting tutorial.

tawk 'bitsanyset($ftpStat, 0x01)' ~/results/faf-exercise_flows.txt | tcol

%dir  flowInd  flowStat            timeFirst             timeLast              duration       numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  vlanID  srcIP          srcIPCC  srcIPOrg             srcPort  dstIP          dstIPCC  dstIPOrg             dstPort  l4Proto  macStat  macPairs  srcMac_dstMac_numP                      srcMacLbl_dstMacLbl     dstPortClassN  dstPortClass  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT    aveIAT    stdIAT    pktps       bytps     pktAsm  bytAsm      tcpFStat  ipMindIPID  ipMaxdIPID  ipMinTTL  ipMaxTTL  ipTTLChg  ipToS  ipFlags  ipOptCnt  ipOptCpCl_Num    ip6OptCntHH_D  ip6OptHH_D             tcpISeqN    tcpPSeqCnt  tcpSeqSntBytes  tcpSeqFaultCnt  tcpPAckCnt  tcpFlwLssAckRcvdBytes  tcpAckFaultCnt  tcpBFlgtMx  tcpInitWinSz  tcpAveWinSz  tcpMinWinSz  tcpMaxWinSz  tcpWinSzDwnCnt  tcpWinSzUpCnt  tcpWinSzChgDirCnt  tcpWinSzThRt  tcpFlags  tcpAnomaly  tcpOptPktCnt  tcpOptCnt  tcpOptions  tcpMSS  tcpWS  tcpMPTBF  tcpMPF  tcpMPAID  tcpMPDSSF  tcpTmS  tcpTmER  tcpEcI  tcpUtm    tcpBtm       tcpSSASAATrip  tcpRTTAckTripMin  tcpRTTAckTripMax  tcpRTTAckTripAve  tcpRTTAckTripJitAve  tcpRTTSseqAA  tcpRTTAckJitAve  tcpStatesAFlags  icmpStat  icmpTCcnt  icmpBFTypH_TypL_Code          icmpTmGtw   icmpEchoSuccRatio  icmpPFindex  ftpStat  ftpCDFindex  ftpCC                          ftpRC                            ftpNumUser  ftpUser      ftpNumPass  ftpPass    ftpNumCP  ftpCP
A     35       0x0400000000004000  1258594162.928342000  1258594185.618346000  22.690004000   1           3        eth:ipv4:tcp  00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800           192.168.1.105  07       "Private network"    49329    143.166.11.10  us       "Dell Technologies"  21       6        0x00     1         00:08:74:38:01:b4_00:19:e3:e7:5d:23_11  DellInc,US_AppleInc,US  21             ftp           11          11           92           1231          0         24        8.363636    10.15629    0       21.78007  2.062728  6.53995   0.484795    4.054649  0       -0.8609222  0x0811    1           2328        128       128       0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2427598871  10          92              0               10          1231                   0               24          8192          62176.56     8192         64860        8               1              9                  0             0x001a    0x0000      1             4          0x00000016  1460    1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0.08025197     0.07749402        306.0649          29.85101          91.8391              0.08094997    0                0x02             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x09     36           USER;PASS;TYPE;PASV;SIZE;RETR                                   1           "anonymous"  1           "IEUser@"  2         "I";"/video/R79733.EXE"
B     35       0x0400000000004001  1258594163.008594000  1258594491.683288000  328.674694000  1           3        eth:ipv4:tcp  00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800           143.166.11.10  us       "Dell Technologies"  21       192.168.1.105  07       "Private network"    49329    6        0x00     1         00:19:e3:e7:5d:23_00:08:74:38:01:b4_11  AppleInc,US_DellInc,US  21             ftp           11          11           1231         92            0         950       111.9091    303.9246    0       306.2558  29.87952  91.89713  0.03346774  3.745345  0       0.8609222   0x0811    2732        26560       239       239       0         0x00   0x0044   0         0x00_0x00000000  0_0            0x00000000_0x00000000  365320932   11          1231            0               11          92                     0               971         4140          4214.603     4140         4232         0               6              1                  0             0x061e    0x0000      1             2          0x00000014  1380    1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0.000698       0.000449984       0.194089          0.04303963        0.07786669           29.89405      91.83913         0x42             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x09     36                                          220;331;230;200;227;213;125;226  0                        0                      1         "125 Data connection already open; Transfer startin"

The packet mode provides now more info per packet. Now, the evolution of the anomaly bits, packet lengths, the seq/ack numbers, checksums and window size can be extracted on a packet per packet basis and directly fed into sequence analysis algorithms.

tawk 'flow(35)' ~/results/faf-exercise_packets.txt | tcol

%pktNo  flowInd  flowStat            time                  pktIAT         pktTrip        flowDuration   numHdrs  hdrDesc       vlanID  srcMac             dstMac             ethType  srcIP          srcIPCC  srcIPOrg           srcPort  dstIP          dstIPCC  dstIPOrg           dstPort  l4Proto  srcMacLbl    dstMacLbl    dstPortClassN  dstPortClass  pktLen  udpLen  snapL4Len  snapL7Len  l7Len  ipToS  ipID   ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq   ack   seqMax  seqDiff  ackDiff  seqLen  ackLen  seqFlowLen  ackFlowLen  tcpMLen  tcpBFlgt  tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpWS  tcpMSS  tcpTmS  tcpTmER  tcpMPTyp  tcpMPF  tcpMPAID  tcpMPDSSF  tcpOptLen  tcpOpts                                  tcpStatesAFlags  icmpStat  icmpType  icmpCode  icmpID  icmpSeq  icmpPFindex  ftpStat  l7Content
1266    35       0x0400000000004000  1258594162.928342000  0.000000000    0.000000000    0.000000000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           66      0       28         0          0      0x00   16230  0         0x4000  128    0x5ea0       0x5ea0       0x7ccd       0x7ccd       0x0040   0                       0                     0                 0     0     0       0        0        0       0       0           0           0        0         0x0010    0x0002    0x0000      8192    0      1460    0       0        0         0x00    0         0x00       8          0x02;0x04;0x05;0xb4;0x01;0x01;0x04;0x02  0x00             0x00                                                        0x01
1267    35       0x0400000000004001  1258594163.008594000  0.000000000    0.080251968    0.000000000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           62      0       28         0          0      0x00   55468  0         0x4000  239    0x5659       0x5659       0x1d37       0x1d37       0x0040   0                       0                     0                 0     1     0       0        0        0       0       0           0           0        0         0x0010    0x0212    0x0000      4140    0      1380    0       0        0         0x00    0         0x00       8          0x02;0x04;0x05;0x64;0x04;0x02;0x00;0x00  0x00             0x00                                                        0x01
1268    35       0x0400000000004000  1258594163.009292000  0.080950000    0.000698000    0.080950000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16231  1         0x4000  128    0x5ea7       0x5ea7       0x5b79       0x5b79       0x0040   0                       0                     0                 1     1     1       1        0        0       0       0           0           0        0         0x0811    0x0010    0x0000      64860   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01
1269    35       0x0400000000004001  1258594163.087792000  0.079198000    0.078500000    0.079198000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           81      0       47         27         27     0x00   58625  3157      0x4000  239    0x49f1       0x49f1       0xad9d       0xad9d       0x0040   0                       0                     0                 1     1     1       1        0        0       0       0           0           27       27        0x0011    0x0018    0x0000      4140    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     220 Microsoft FTP Service\r\n
1270    35       0x0400000000004000  1258594163.088491000  0.079199000    0.000699000    0.160149000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           74      0       36         16         16     0x00   16243  12        0x4000  128    0x5e8b       0x5e8b       0xd384       0xd384       0x0040   0                       0                     0                 1     28    1       0        27       0       27      0           27          16       16        0x0011    0x0018    0x0000      64833   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     USER anonymous\r\n
1271    35       0x0400000000004001  1258594163.166256000  0.078464000    0.077765000    0.157662000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           126     0       92         72         72     0x00   61580  2955      0x4000  239    0x3e39       0x3e39       0xf987       0xf987       0x0040   0                       0                     0                 28    17    28      27       16       27      16      27          16          99       72        0x0811    0x0018    0x0000      4156    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     331 Anonymous access allowed, send identity (e-mail name) as password.\r\n
1272    35       0x0400000000004000  1258594163.168693000  0.080202000    0.002436992    0.240351000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           72      0       34         14         14     0x00   16244  1         0x4000  128    0x5e8c       0x5e8c       0x5f70       0x5f70       0x0040   0                       0                     0                 17    100   17      16       72       16      72      16          99          30       14        0x0011    0x0018    0x0000      64761   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     PASS IEUser@\r\n
1273    35       0x0400000000004001  1258594163.247178000  0.080922000    0.078485008    0.238584000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           1004    0       970        950        950    0x00   64425  2845      0x4000  239    0x2fae       0x2fae       0x41de       0x41de       0x0040   0                       0                     0                 100   31    100     72       14       72      14      99          30          1049     950       0x0811    0x0018    0x0000      4170    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     230-Welcome to the Dell FTP site. A service of Dell Inc., Round Rock, Texas.\r\n    For information about DELL, call +1 800 999 3355 All transfers are logged with\r\n    your host name and email address. If you don't like this policy please disconnect now.\r\n    Please be advised that use constitutes consent to monitoring (Elec Comm Priv Act,\r\n    18 USC 2701-2711). Please see the file readme.txt for disclaimers pertaining to this\r\n    service. If your FTP client crashes or hangs shortly after login, try using a dash\r\n    (-) as the first character of your password. This will turn off the informational\r\n    messages which may be confusing your ftp client.\r\n    ********IN CASE OF PROBLEMS*************************\r\n    ** File Content: send EMAIL to dellbbs@dell.com   **\r\n    ** FTP Server: send EMAIL to hostmaster@dell.com  **\r\n    ** WWW Server: send EMAIL to webmaster@dell.com   **\r\n    ****************************************************\r\n
1274    35       0x0400000000004001  1258594163.247187000  0.000009000    0.078494016    0.238593000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           75      0       41         21         21     0x00   64426  1         0x4000  239    0x334e       0x334e       0x2a2a       0x2a2a       0x0040   0                       0                     0                 1050  31    1050    950      0        950     0       1049        30          1070     971       0x0811    0x0018    0x0000      4170    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     230 User logged in.\r\n
1275    35       0x0400000000004000  1258594163.247637000  0.078944000    0.000449984    0.319295000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16253  9         0x4000  128    0x5e91       0x5e91       0x5b5b       0x5b5b       0x0040   0                       0                     0                 31    1071  31      14       971      14      971     30          1070        30       0         0x0011    0x0010    0x0000      63790   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01
1276    35       0x0400000000004000  1258594163.249385000  0.001748000    0.002197984    0.321043000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           66      0       28         8          8      0x00   16254  1         0x4000  128    0x5e88       0x5e88       0x8959       0x8959       0x0040   0                       0                     0                 31    1071  31      0        0        0       0       30          1070        38       8         0x0811    0x0018    0x0000      63790   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     TYPE I\r\n
1277    35       0x0400000000004001  1258594163.327121000  0.079934000    0.077736000    0.318527000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           74      0       40         20         20     0x00   1622   -62804    0x4000  239    0x28a4       0x28a4       0xb130       0xb130       0x0044   0                       0                     0                 1071  39    1071    21       8        21      8       1070        38          1090     20        0x0811    0x0018    0x0000      4178    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     200 Type set to I.\r\n
1278    35       0x0400000000004000  1258594163.327845000  0.078460000    0.000724000    0.399503000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       26         6          6      0x00   16255  1         0x4000  128    0x5e89       0x5e89       0xaaa3       0xaaa3       0x0040   0                       0                     0                 39    1091  39      8        20       8       20      38          1090        44       6         0x0011    0x0018    0x0000      63770   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     PASV\r\n
1279    35       0x0400000000004001  1258594163.407582000  0.080461000    0.079737024    0.398988000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           104     0       70         50         50     0x00   5259   3637      0x4000  239    0x1a51       0x1a51       0xbf53       0xbf53       0x0040   0                       0                     0                 1091  45    1091    20       6        20      6       1090        44          1140     50        0x0811    0x0018    0x0000      4184    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     227 Entering Passive Mode (143,166,11,10,251,78)\r\n
1283    35       0x0400000000004000  1258594163.487490000  0.159645000    0.079907968    0.559148000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           82      0       44         24         24     0x00   16267  12        0x4000  128    0x5e6b       0x5e6b       0xf13a       0xf13a       0x0040   0                       0                     0                 45    1141  45      6        50       6       50      44          1140        68       24        0x0011    0x0018    0x0000      63720   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     SIZE /video/R79733.EXE\r\n
1284    35       0x0400000000004001  1258594163.565990000  0.158408000    0.078500032    0.557396000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           67      0       33         13         13     0x00   11024  5765      0x4000  239    0x03f1       0x03f1       0x049e       0x049e       0x0040   0                       0                     0                 1141  69    1141    50       24       50      24      1140        68          1153     13        0x0811    0x0018    0x0000      4208    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     213 4255056\r\n
1285    35       0x0400000000004000  1258594163.566694000  0.079204000    0.000704000    0.638352000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           82      0       44         24         24     0x00   16268  1         0x4000  128    0x5e6a       0x5e6a       0xf819       0xf819       0x0040   0                       0                     0                 69    1154  69      24       13       24      13      68          1153        92       24        0x0011    0x0018    0x0000      63707   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     RETR /video/R79733.EXE\r\n
1286    35       0x0400000000004001  1258594163.644188000  0.078198000    0.077494016    0.635594000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           108     0       74         54         54     0x00   14255  3231      0x4000  239    0xf728       0xf728       0xb8e3       0xb8e3       0x0040   0                       0                     0                 1154  93    1154    13       24       13      24      1153        92          1207     54        0x0811    0x0018    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     125 Data connection already open; Transfer starting.\r\n
1303    35       0x0400000000004000  1258594163.838277000  0.271583000    0.194088960    0.909935000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16289  21        0x4000  128    0x5e6d       0x5e6d       0x5b1d       0x5b1d       0x0040   0                       0                     0                 93    1208  93      24       54       24      54      92          1207        92       0         0x0011    0x0010    0x0000      63653   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09
5898    35       0x0400000000004001  1258594185.427515000  21.783327000   21.589238016   22.418921000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           78      0       44         24         24     0x00   40815  26560     0x4000  239    0x8f86       0x8f86       0x7425       0x7425       0x0040   0                       0                     0                 1208  93    1208    54       0        54      0       1207        92          1231     24        0x0811    0x0018    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     226 Transfer complete.\r\n
5900    35       0x0400000000004000  1258594185.618346000  21.780069000   0.190830976    22.690004000   3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   18617  2328      0x4000  128    0x5555       0x5555       0x5b1d       0x5b1d       0x0040   0                       0                     0                 93    1232  93      0        24       0       24      92          1231        92       0         0x0011    0x0010    0x0000      63629   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09
5902    35       0x0400000000004001  1258594491.683288000  306.255773000  306.064942016  328.674694000  3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           54      0       20         0          0      0x00   49361  8546      0x4000  239    0x6e3c       0x6e3c       0x431f       0x431f       0x0040   0                       0                     0                 1232  93    1232    24       0        24      0       1231        92          1231     0         0x0811    0x0414    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x42             0x00                                                        0x09

Changing L7 output format to hex

The configuration of the packet mode currently resides at compile-time in a header file: main.h. This will change in future and bring more flexibility to the packet mode. You can either switch on/off the packet number or the output type of layer 7 content. Both may be switched on simultaneously, in which case the human readable output is appended after the hexadecimal output. The prepended 0x facilitates the post-processing with command line scripts. Nevertheless, it may be simpler for the human eye to remove these characters, then set SPKTMD_PCNTH_PREF to "". Also the separator can be changed, default is " ". The start of the printout can be chosen with SPKTMD_PCNTL, default is the layer 7 header.

tranalyzer2

vi src/main.h

/* ========================================================================== */
/* ------------------------ USER CONFIGURATION FLAGS ------------------------ */
/* ========================================================================== */
...

// Packet mode (-s option)
#define SPKTMD_PKTNO    1 // Whether or not to print the packet number
#define SPKTMD_PCNTC    1 // Whether or not to print L7 content as characters
#define SPKTMD_PCNTH    0 // Whether or not to print L7 content as hex
#define SPKTMD_PCNTL    4 // 0: Print the full payload of the packet
                          // 1: Print payload from L2
                          // 2: Print payload from L3
                          // 3: Print payload from L4
                          // 4: Print payload from L7

#define SPKTMD_BOPS  0x00 // Operations on content
                          //    0x00: MSB, no bit inverse, no shift
                          //    0x01: LSB, Bit inverse
                          //    0x02: Nibble SWAP
                          //    0x10: Shift right
                          //    0x20: if 0x10: shift from last byte into extra trailing byte
...
#define SPKTMD_BSHFT_POS 5 // Bitshift byte pos start
#define SPKTMD_BSHFT     2 // Bitshift

/* +++++++++++++++++++++ ENV / RUNTIME - conf Variables +++++++++++++++++++++ */

// Configure packet mode as hex
#define SPKTMD_PCNTH_PREF "0x" // Prefix to add to every byte ("" -> ab cd instead of 0xab 0xcd)
#define SPKTMD_PCNTH_SEP  " "  // Byte separator ("," -> 0xab,0xcd instead of 0xab 0xcd)

/* ========================================================================== */
/* ------------------------- DO NOT EDIT BELOW HERE ------------------------- */
/* ========================================================================== */

Now switch from human readable to hexadecimal values using t2conf, recompile the core and rerun t2:

t2conf tranalyzer2 -D SPKTMD_PCNTC=0 -D SPKTMD_PCNTH=1 && t2build tranalyzer2

t2 -r ~/data/faf-exercise.pcap -w ~/results -s

If we select our FTP flow again by the flowInd, we now find the L7 output in hex. The format enables you to directly read the L7 binary content with tawk without recoding from text.

tawk 'flow(35)' ~/results/faf-exercise_packets.txt | tcol

%pktNo  flowInd  flowStat            time                  pktIAT         pktTrip        flowDuration   numHdrs  hdrDesc       vlanID  srcMac             dstMac             ethType  srcIP          srcIPCC  srcIPOrg           srcPort  dstIP          dstIPCC  dstIPOrg           dstPort  l4Proto  srcMacLbl    dstMacLbl    dstPortClassN  dstPortClass  pktLen  udpLen  snapL4Len  snapL7Len  l7Len  ipToS  ipID   ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq   ack   seqMax  seqDiff  ackDiff  seqLen  ackLen  seqFlowLen  ackFlowLen  tcpMLen  tcpBFlgt  tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpWS  tcpMSS  tcpTmS  tcpTmER  tcpMPTyp  tcpMPF  tcpMPAID  tcpMPDSSF  tcpOptLen  tcpOpts                                  tcpStatesAFlags  icmpStat  icmpType  icmpCode  icmpID  icmpSeq  icmpPFindex  ftpStat  l7HexContent
1266    35       0x0400000000004000  1258594162.928342000  0.000000000    0.000000000    0.000000000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           66      0       28         0          0      0x00   16230  0         0x4000  128    0x5ea0       0x5ea0       0x7ccd       0x7ccd       0x0040   0                       0                     0                 0     0     0       0        0        0       0       0           0           0        0         0x0010    0x0002    0x0000      8192    0      1460    0       0        0         0x00    0         0x00       8          0x02;0x04;0x05;0xb4;0x01;0x01;0x04;0x02  0x00             0x00                                                        0x01
1267    35       0x0400000000004001  1258594163.008594000  0.000000000    0.080251968    0.000000000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           62      0       28         0          0      0x00   55468  0         0x4000  239    0x5659       0x5659       0x1d37       0x1d37       0x0040   0                       0                     0                 0     1     0       0        0        0       0       0           0           0        0         0x0010    0x0212    0x0000      4140    0      1380    0       0        0         0x00    0         0x00       8          0x02;0x04;0x05;0x64;0x04;0x02;0x00;0x00  0x00             0x00                                                        0x01
1268    35       0x0400000000004000  1258594163.009292000  0.080950000    0.000698000    0.080950000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16231  1         0x4000  128    0x5ea7       0x5ea7       0x5b79       0x5b79       0x0040   0                       0                     0                 1     1     1       1        0        0       0       0           0           0        0         0x0811    0x0010    0x0000      64860   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01
1269    35       0x0400000000004001  1258594163.087792000  0.079198000    0.078500000    0.079198000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           81      0       47         27         27     0x00   58625  3157      0x4000  239    0x49f1       0x49f1       0xad9d       0xad9d       0x0040   0                       0                     0                 1     1     1       1        0        0       0       0           0           27       27        0x0011    0x0018    0x0000      4140    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x32 0x32 0x30 0x20 0x4d 0x69 0x63 0x72 0x6f 0x73 0x6f 0x66 0x74 0x20 0x46 0x54 0x50 0x20 0x53 0x65 0x72 0x76 0x69 0x63 0x65 0x0d 0x0a
1270    35       0x0400000000004000  1258594163.088491000  0.079199000    0.000699000    0.160149000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           74      0       36         16         16     0x00   16243  12        0x4000  128    0x5e8b       0x5e8b       0xd384       0xd384       0x0040   0                       0                     0                 1     28    1       0        27       0       27      0           27          16       16        0x0011    0x0018    0x0000      64833   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x55 0x53 0x45 0x52 0x20 0x61 0x6e 0x6f 0x6e 0x79 0x6d 0x6f 0x75 0x73 0x0d 0x0a
1271    35       0x0400000000004001  1258594163.166256000  0.078464000    0.077765000    0.157662000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           126     0       92         72         72     0x00   61580  2955      0x4000  239    0x3e39       0x3e39       0xf987       0xf987       0x0040   0                       0                     0                 28    17    28      27       16       27      16      27          16          99       72        0x0811    0x0018    0x0000      4156    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x33 0x33 0x31 0x20 0x41 0x6e 0x6f 0x6e 0x79 0x6d 0x6f 0x75 0x73 0x20 0x61 0x63 0x63 0x65 0x73 0x73 0x20 0x61 0x6c 0x6c 0x6f 0x77 0x65 0x64 0x2c 0x20 0x73 0x65 0x6e 0x64 0x20 0x69 0x64 0x65 0x6e 0x74 0x69 0x74 0x79 0x20 0x28 0x65 0x2d 0x6d 0x61 0x69 0x6c 0x20 0x6e 0x61 0x6d 0x65 0x29 0x20 0x61 0x73 0x20 0x70 0x61 0x73 0x73 0x77 0x6f 0x72 0x64 0x2e 0x0d 0x0a
1272    35       0x0400000000004000  1258594163.168693000  0.080202000    0.002436992    0.240351000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           72      0       34         14         14     0x00   16244  1         0x4000  128    0x5e8c       0x5e8c       0x5f70       0x5f70       0x0040   0                       0                     0                 17    100   17      16       72       16      72      16          99          30       14        0x0011    0x0018    0x0000      64761   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x50 0x41 0x53 0x53 0x20 0x49 0x45 0x55 0x73 0x65 0x72 0x40 0x0d 0x0a
1273    35       0x0400000000004001  1258594163.247178000  0.080922000    0.078485008    0.238584000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           1004    0       970        950        950    0x00   64425  2845      0x4000  239    0x2fae       0x2fae       0x41de       0x41de       0x0040   0                       0                     0                 100   31    100     72       14       72      14      99          30          1049     950       0x0811    0x0018    0x0000      4170    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x32 0x33 0x30 0x2d 0x57 0x65 0x6c 0x63 0x6f 0x6d 0x65 0x20 0x74 0x6f 0x20 0x74 0x68 0x65 0x20 0x44 0x65 0x6c 0x6c 0x20 0x46 0x54 0x50 0x20 0x73 0x69 0x74 0x65 0x2e 0x20 0x41 0x20 0x73 0x65 0x72 0x76 0x69 0x63 0x65 0x20 0x6f 0x66 0x20 0x44 0x65 0x6c 0x6c 0x20 0x49 0x6e 0x63 0x2e 0x2c 0x20 0x52 0x6f 0x75 0x6e 0x64 0x20 0x52 0x6f 0x63 0x6b 0x2c 0x20 0x54 0x65 0x78 0x61 0x73 0x2e 0x0d 0x0a 0x20 0x20 0x20 0x20 0x46 0x6f 0x72 0x20 0x69 0x6e 0x66 0x6f 0x72 0x6d 0x61 0x74 0x69 0x6f 0x6e 0x20 0x61 0x62 0x6f 0x75 0x74 0x20 0x44 0x45 0x4c 0x4c 0x2c 0x20 0x63 0x61 0x6c 0x6c 0x20 0x2b 0x31 0x20 0x38 0x30 0x30 0x20 0x39 0x39 0x39 0x20 0x33 0x33 0x35 0x35 0x20 0x41 0x6c 0x6c 0x20 0x74 0x72 0x61 0x6e 0x73 0x66 0x65 0x72 0x73 0x20 0x61 0x72 0x65 0x20 0x6c 0x6f 0x67 0x67 0x65 0x64 0x20 0x77 0x69 0x74 0x68 0x0d 0x0a 0x20 0x20 0x20 0x20 0x79 0x6f 0x75 0x72 0x20 0x68 0x6f 0x73 0x74 0x20 0x6e 0x61 0x6d 0x65 0x20 0x61 0x6e 0x64 0x20 0x65 0x6d 0x61 0x69 0x6c 0x20 0x61 0x64 0x64 0x72 0x65 0x73 0x73 0x2e 0x20 0x49 0x66 0x20 0x79 0x6f 0x75 0x20 0x64 0x6f 0x6e 0x27 0x74 0x20 0x6c 0x69 0x6b 0x65 0x20 0x74 0x68 0x69 0x73 0x20 0x70 0x6f 0x6c 0x69 0x63 0x79 0x20 0x70 0x6c 0x65 0x61 0x73 0x65 0x20 0x64 0x69 0x73 0x63 0x6f 0x6e 0x6e 0x65 0x63 0x74 0x20 0x6e 0x6f 0x77 0x2e 0x0d 0x0a 0x20 0x20 0x20 0x20 0x50 0x6c 0x65 0x61 0x73 0x65 0x20 0x62 0x65 0x20 0x61 0x64 0x76 0x69 0x73 0x65 0x64 0x20 0x74 0x68 0x61 0x74 0x20 0x75 0x73 0x65 0x20 0x63 0x6f 0x6e 0x73 0x74 0x69 0x74 0x75 0x74 0x65 0x73 0x20 0x63 0x6f 0x6e 0x73 0x65 0x6e 0x74 0x20 0x74 0x6f 0x20 0x6d 0x6f 0x6e 0x69 0x74 0x6f 0x72 0x69 0x6e 0x67 0x20 0x28 0x45 0x6c 0x65 0x63 0x20 0x43 0x6f 0x6d 0x6d 0x20 0x50 0x72 0x69 0x76 0x20 0x41 0x63 0x74 0x2c 0x0d 0x0a 0x20 0x20 0x20 0x20 0x31 0x38 0x20 0x55 0x53 0x43 0x20 0x32 0x37 0x30 0x31 0x2d 0x32 0x37 0x31 0x31 0x29 0x2e 0x20 0x50 0x6c 0x65 0x61 0x73 0x65 0x20 0x73 0x65 0x65 0x20 0x74 0x68 0x65 0x20 0x66 0x69 0x6c 0x65 0x20 0x72 0x65 0x61 0x64 0x6d 0x65 0x2e 0x74 0x78 0x74 0x20 0x66 0x6f 0x72 0x20 0x64 0x69 0x73 0x63 0x6c 0x61 0x69 0x6d 0x65 0x72 0x73 0x20 0x70 0x65 0x72 0x74 0x61 0x69 0x6e 0x69 0x6e 0x67 0x20 0x74 0x6f 0x20 0x74 0x68 0x69 0x73 0x0d 0x0a 0x20 0x20 0x20 0x20 0x73 0x65 0x72 0x76 0x69 0x63 0x65 0x2e 0x20 0x49 0x66 0x20 0x79 0x6f 0x75 0x72 0x20 0x46 0x54 0x50 0x20 0x63 0x6c 0x69 0x65 0x6e 0x74 0x20 0x63 0x72 0x61 0x73 0x68 0x65 0x73 0x20 0x6f 0x72 0x20 0x68 0x61 0x6e 0x67 0x73 0x20 0x73 0x68 0x6f 0x72 0x74 0x6c 0x79 0x20 0x61 0x66 0x74 0x65 0x72 0x20 0x6c 0x6f 0x67 0x69 0x6e 0x2c 0x20 0x74 0x72 0x79 0x20 0x75 0x73 0x69 0x6e 0x67 0x20 0x61 0x20 0x64 0x61 0x73 0x68 0x0d 0x0a 0x20 0x20 0x20 0x20 0x28 0x2d 0x29 0x20 0x61 0x73 0x20 0x74 0x68 0x65 0x20 0x66 0x69 0x72 0x73 0x74 0x20 0x63 0x68 0x61 0x72 0x61 0x63 0x74 0x65 0x72 0x20 0x6f 0x66 0x20 0x79 0x6f 0x75 0x72 0x20 0x70 0x61 0x73 0x73 0x77 0x6f 0x72 0x64 0x2e 0x20 0x54 0x68 0x69 0x73 0x20 0x77 0x69 0x6c 0x6c 0x20 0x74 0x75 0x72 0x6e 0x20 0x6f 0x66 0x66 0x20 0x74 0x68 0x65 0x20 0x69 0x6e 0x66 0x6f 0x72 0x6d 0x61 0x74 0x69 0x6f 0x6e 0x61 0x6c 0x0d 0x0a 0x20 0x20 0x20 0x20 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x20 0x77 0x68 0x69 0x63 0x68 0x20 0x6d 0x61 0x79 0x20 0x62 0x65 0x20 0x63 0x6f 0x6e 0x66 0x75 0x73 0x69 0x6e 0x67 0x20 0x79 0x6f 0x75 0x72 0x20 0x66 0x74 0x70 0x20 0x63 0x6c 0x69 0x65 0x6e 0x74 0x2e 0x0d 0x0a 0x20 0x20 0x20 0x20 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x49 0x4e 0x20 0x43 0x41 0x53 0x45 0x20 0x4f 0x46 0x20 0x50 0x52 0x4f 0x42 0x4c 0x45 0x4d 0x53 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x0d 0x0a 0x20 0x20 0x20 0x20 0x2a 0x2a 0x20 0x46 0x69 0x6c 0x65 0x20 0x43 0x6f 0x6e 0x74 0x65 0x6e 0x74 0x3a 0x20 0x73 0x65 0x6e 0x64 0x20 0x45 0x4d 0x41 0x49 0x4c 0x20 0x74 0x6f 0x20 0x64 0x65 0x6c 0x6c 0x62 0x62 0x73 0x40 0x64 0x65 0x6c 0x6c 0x2e 0x63 0x6f 0x6d 0x20 0x20 0x20 0x2a 0x2a 0x0d 0x0a 0x20 0x20 0x20 0x20 0x2a 0x2a 0x20 0x46 0x54 0x50 0x20 0x53 0x65 0x72 0x76 0x65 0x72 0x3a 0x20 0x73 0x65 0x6e 0x64 0x20 0x45 0x4d 0x41 0x49 0x4c 0x20 0x74 0x6f 0x20 0x68 0x6f 0x73 0x74 0x6d 0x61 0x73 0x74 0x65 0x72 0x40 0x64 0x65 0x6c 0x6c 0x2e 0x63 0x6f 0x6d 0x20 0x20 0x2a 0x2a 0x0d 0x0a 0x20 0x20 0x20 0x20 0x2a 0x2a 0x20 0x57 0x57 0x57 0x20 0x53 0x65 0x72 0x76 0x65 0x72 0x3a 0x20 0x73 0x65 0x6e 0x64 0x20 0x45 0x4d 0x41 0x49 0x4c 0x20 0x74 0x6f 0x20 0x77 0x65 0x62 0x6d 0x61 0x73 0x74 0x65 0x72 0x40 0x64 0x65 0x6c 0x6c 0x2e 0x63 0x6f 0x6d 0x20 0x20 0x20 0x2a 0x2a 0x0d 0x0a 0x20 0x20 0x20 0x20 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x2a 0x0d 0x0a
1274    35       0x0400000000004001  1258594163.247187000  0.000009000    0.078494016    0.238593000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           75      0       41         21         21     0x00   64426  1         0x4000  239    0x334e       0x334e       0x2a2a       0x2a2a       0x0040   0                       0                     0                 1050  31    1050    950      0        950     0       1049        30          1070     971       0x0811    0x0018    0x0000      4170    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x32 0x33 0x30 0x20 0x55 0x73 0x65 0x72 0x20 0x6c 0x6f 0x67 0x67 0x65 0x64 0x20 0x69 0x6e 0x2e 0x0d 0x0a
1275    35       0x0400000000004000  1258594163.247637000  0.078944000    0.000449984    0.319295000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16253  9         0x4000  128    0x5e91       0x5e91       0x5b5b       0x5b5b       0x0040   0                       0                     0                 31    1071  31      14       971      14      971     30          1070        30       0         0x0011    0x0010    0x0000      63790   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01
1276    35       0x0400000000004000  1258594163.249385000  0.001748000    0.002197984    0.321043000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           66      0       28         8          8      0x00   16254  1         0x4000  128    0x5e88       0x5e88       0x8959       0x8959       0x0040   0                       0                     0                 31    1071  31      0        0        0       0       30          1070        38       8         0x0811    0x0018    0x0000      63790   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x54 0x59 0x50 0x45 0x20 0x49 0x0d 0x0a
1277    35       0x0400000000004001  1258594163.327121000  0.079934000    0.077736000    0.318527000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           74      0       40         20         20     0x00   1622   -62804    0x4000  239    0x28a4       0x28a4       0xb130       0xb130       0x0044   0                       0                     0                 1071  39    1071    21       8        21      8       1070        38          1090     20        0x0811    0x0018    0x0000      4178    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x32 0x30 0x30 0x20 0x54 0x79 0x70 0x65 0x20 0x73 0x65 0x74 0x20 0x74 0x6f 0x20 0x49 0x2e 0x0d 0x0a
1278    35       0x0400000000004000  1258594163.327845000  0.078460000    0.000724000    0.399503000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       26         6          6      0x00   16255  1         0x4000  128    0x5e89       0x5e89       0xaaa3       0xaaa3       0x0040   0                       0                     0                 39    1091  39      8        20       8       20      38          1090        44       6         0x0011    0x0018    0x0000      63770   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     0x50 0x41 0x53 0x56 0x0d 0x0a
1279    35       0x0400000000004001  1258594163.407582000  0.080461000    0.079737024    0.398988000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           104     0       70         50         50     0x00   5259   3637      0x4000  239    0x1a51       0x1a51       0xbf53       0xbf53       0x0040   0                       0                     0                 1091  45    1091    20       6        20      6       1090        44          1140     50        0x0811    0x0018    0x0000      4184    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     0x32 0x32 0x37 0x20 0x45 0x6e 0x74 0x65 0x72 0x69 0x6e 0x67 0x20 0x50 0x61 0x73 0x73 0x69 0x76 0x65 0x20 0x4d 0x6f 0x64 0x65 0x20 0x28 0x31 0x34 0x33 0x2c 0x31 0x36 0x36 0x2c 0x31 0x31 0x2c 0x31 0x30 0x2c 0x32 0x35 0x31 0x2c 0x37 0x38 0x29 0x0d 0x0a
1283    35       0x0400000000004000  1258594163.487490000  0.159645000    0.079907968    0.559148000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           82      0       44         24         24     0x00   16267  12        0x4000  128    0x5e6b       0x5e6b       0xf13a       0xf13a       0x0040   0                       0                     0                 45    1141  45      6        50       6       50      44          1140        68       24        0x0011    0x0018    0x0000      63720   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     0x53 0x49 0x5a 0x45 0x20 0x2f 0x76 0x69 0x64 0x65 0x6f 0x2f 0x52 0x37 0x39 0x37 0x33 0x33 0x2e 0x45 0x58 0x45 0x0d 0x0a
1284    35       0x0400000000004001  1258594163.565990000  0.158408000    0.078500032    0.557396000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           67      0       33         13         13     0x00   11024  5765      0x4000  239    0x03f1       0x03f1       0x049e       0x049e       0x0040   0                       0                     0                 1141  69    1141    50       24       50      24      1140        68          1153     13        0x0811    0x0018    0x0000      4208    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     0x32 0x31 0x33 0x20 0x34 0x32 0x35 0x35 0x30 0x35 0x36 0x0d 0x0a
1285    35       0x0400000000004000  1258594163.566694000  0.079204000    0.000704000    0.638352000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           82      0       44         24         24     0x00   16268  1         0x4000  128    0x5e6a       0x5e6a       0xf819       0xf819       0x0040   0                       0                     0                 69    1154  69      24       13       24      13      68          1153        92       24        0x0011    0x0018    0x0000      63707   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     0x52 0x45 0x54 0x52 0x20 0x2f 0x76 0x69 0x64 0x65 0x6f 0x2f 0x52 0x37 0x39 0x37 0x33 0x33 0x2e 0x45 0x58 0x45 0x0d 0x0a
1286    35       0x0400000000004001  1258594163.644188000  0.078198000    0.077494016    0.635594000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           108     0       74         54         54     0x00   14255  3231      0x4000  239    0xf728       0xf728       0xb8e3       0xb8e3       0x0040   0                       0                     0                 1154  93    1154    13       24       13      24      1153        92          1207     54        0x0811    0x0018    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     0x31 0x32 0x35 0x20 0x44 0x61 0x74 0x61 0x20 0x63 0x6f 0x6e 0x6e 0x65 0x63 0x74 0x69 0x6f 0x6e 0x20 0x61 0x6c 0x72 0x65 0x61 0x64 0x79 0x20 0x6f 0x70 0x65 0x6e 0x3b 0x20 0x54 0x72 0x61 0x6e 0x73 0x66 0x65 0x72 0x20 0x73 0x74 0x61 0x72 0x74 0x69 0x6e 0x67 0x2e 0x0d 0x0a
1303    35       0x0400000000004000  1258594163.838277000  0.271583000    0.194088960    0.909935000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16289  21        0x4000  128    0x5e6d       0x5e6d       0x5b1d       0x5b1d       0x0040   0                       0                     0                 93    1208  93      24       54       24      54      92          1207        92       0         0x0011    0x0010    0x0000      63653   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09
5898    35       0x0400000000004001  1258594185.427515000  21.783327000   21.589238016   22.418921000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           78      0       44         24         24     0x00   40815  26560     0x4000  239    0x8f86       0x8f86       0x7425       0x7425       0x0040   0                       0                     0                 1208  93    1208    54       0        54      0       1207        92          1231     24        0x0811    0x0018    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     0x32 0x32 0x36 0x20 0x54 0x72 0x61 0x6e 0x73 0x66 0x65 0x72 0x20 0x63 0x6f 0x6d 0x70 0x6c 0x65 0x74 0x65 0x2e 0x0d 0x0a
5900    35       0x0400000000004000  1258594185.618346000  21.780069000   0.190830976    22.690004000   3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   18617  2328      0x4000  128    0x5555       0x5555       0x5b1d       0x5b1d       0x0040   0                       0                     0                 93    1232  93      0        24       0       24      92          1231        92       0         0x0011    0x0010    0x0000      63629   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09
5902    35       0x0400000000004001  1258594491.683288000  306.255773000  306.064942016  328.674694000  3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           54      0       20         0          0      0x00   49361  8546      0x4000  239    0x6e3c       0x6e3c       0x431f       0x431f       0x0040   0                       0                     0                 1232  93    1232    24       0        24      0       1231        92          1231     0         0x0811    0x0414    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x42             0x00                                                        0x09

Now try to remove the 0x prefix:

t2conf tranalyzer2 -D SPKTMD_PCNTH_PREF="" && t2build tranalyzer2

t2 -r ~/data/faf-exercise.pcap -w ~/results -s

See? The 0x is gone.

tawk 'flow(35)' ~/results/faf-exercise_packets.txt | tcol

%pktNo  flowInd  flowStat            time                  pktIAT         pktTrip        flowDuration   numHdrs  hdrDesc       vlanID  srcMac             dstMac             ethType  srcIP          srcIPCC  srcIPOrg           srcPort  dstIP          dstIPCC  dstIPOrg           dstPort  l4Proto  srcMacLbl    dstMacLbl    dstPortClassN  dstPortClass  pktLen  udpLen  snapL4Len  snapL7Len  l7Len  ipToS  ipID   ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq   ack   seqMax  seqDiff  ackDiff  seqLen  ackLen  seqFlowLen  ackFlowLen  tcpMLen  tcpBFlgt  tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpWS  tcpMSS  tcpTmS  tcpTmER  tcpMPTyp  tcpMPF  tcpMPAID  tcpMPDSSF  tcpOptLen  tcpOpts                                  tcpStatesAFlags  icmpStat  icmpType  icmpCode  icmpID  icmpSeq  icmpPFindex  ftpStat  l7HexContent
1266    35       0x0400000000004000  1258594162.928342000  0.000000000    0.000000000    0.000000000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           66      0       28         0          0      0x00   16230  0         0x4000  128    0x5ea0       0x5ea0       0x7ccd       0x7ccd       0x0040   0                       0                     0                 0     0     0       0        0        0       0       0           0           0        0         0x0010    0x0002    0x0000      8192    0      1460    0       0        0         0x00    0         0x00       8          0x02;0x04;0x05;0xb4;0x01;0x01;0x04;0x02  0x00             0x00                                                        0x01
1267    35       0x0400000000004001  1258594163.008594000  0.000000000    0.080251968    0.000000000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           62      0       28         0          0      0x00   55468  0         0x4000  239    0x5659       0x5659       0x1d37       0x1d37       0x0040   0                       0                     0                 0     1     0       0        0        0       0       0           0           0        0         0x0010    0x0212    0x0000      4140    0      1380    0       0        0         0x00    0         0x00       8          0x02;0x04;0x05;0x64;0x04;0x02;0x00;0x00  0x00             0x00                                                        0x01
1268    35       0x0400000000004000  1258594163.009292000  0.080950000    0.000698000    0.080950000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16231  1         0x4000  128    0x5ea7       0x5ea7       0x5b79       0x5b79       0x0040   0                       0                     0                 1     1     1       1        0        0       0       0           0           0        0         0x0811    0x0010    0x0000      64860   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01
1269    35       0x0400000000004001  1258594163.087792000  0.079198000    0.078500000    0.079198000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           81      0       47         27         27     0x00   58625  3157      0x4000  239    0x49f1       0x49f1       0xad9d       0xad9d       0x0040   0                       0                     0                 1     1     1       1        0        0       0       0           0           27       27        0x0011    0x0018    0x0000      4140    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     32 32 30 20 4d 69 63 72 6f 73 6f 66 74 20 46 54 50 20 53 65 72 76 69 63 65 0d 0a
1270    35       0x0400000000004000  1258594163.088491000  0.079199000    0.000699000    0.160149000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           74      0       36         16         16     0x00   16243  12        0x4000  128    0x5e8b       0x5e8b       0xd384       0xd384       0x0040   0                       0                     0                 1     28    1       0        27       0       27      0           27          16       16        0x0011    0x0018    0x0000      64833   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     55 53 45 52 20 61 6e 6f 6e 79 6d 6f 75 73 0d 0a
1271    35       0x0400000000004001  1258594163.166256000  0.078464000    0.077765000    0.157662000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           126     0       92         72         72     0x00   61580  2955      0x4000  239    0x3e39       0x3e39       0xf987       0xf987       0x0040   0                       0                     0                 28    17    28      27       16       27      16      27          16          99       72        0x0811    0x0018    0x0000      4156    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     33 33 31 20 41 6e 6f 6e 79 6d 6f 75 73 20 61 63 63 65 73 73 20 61 6c 6c 6f 77 65 64 2c 20 73 65 6e 64 20 69 64 65 6e 74 69 74 79 20 28 65 2d 6d 61 69 6c 20 6e 61 6d 65 29 20 61 73 20 70 61 73 73 77 6f 72 64 2e 0d 0a
1272    35       0x0400000000004000  1258594163.168693000  0.080202000    0.002436992    0.240351000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           72      0       34         14         14     0x00   16244  1         0x4000  128    0x5e8c       0x5e8c       0x5f70       0x5f70       0x0040   0                       0                     0                 17    100   17      16       72       16      72      16          99          30       14        0x0011    0x0018    0x0000      64761   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     50 41 53 53 20 49 45 55 73 65 72 40 0d 0a
1273    35       0x0400000000004001  1258594163.247178000  0.080922000    0.078485008    0.238584000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           1004    0       970        950        950    0x00   64425  2845      0x4000  239    0x2fae       0x2fae       0x41de       0x41de       0x0040   0                       0                     0                 100   31    100     72       14       72      14      99          30          1049     950       0x0811    0x0018    0x0000      4170    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     32 33 30 2d 57 65 6c 63 6f 6d 65 20 74 6f 20 74 68 65 20 44 65 6c 6c 20 46 54 50 20 73 69 74 65 2e 20 41 20 73 65 72 76 69 63 65 20 6f 66 20 44 65 6c 6c 20 49 6e 63 2e 2c 20 52 6f 75 6e 64 20 52 6f 63 6b 2c 20 54 65 78 61 73 2e 0d 0a 20 20 20 20 46 6f 72 20 69 6e 66 6f 72 6d 61 74 69 6f 6e 20 61 62 6f 75 74 20 44 45 4c 4c 2c 20 63 61 6c 6c 20 2b 31 20 38 30 30 20 39 39 39 20 33 33 35 35 20 41 6c 6c 20 74 72 61 6e 73 66 65 72 73 20 61 72 65 20 6c 6f 67 67 65 64 20 77 69 74 68 0d 0a 20 20 20 20 79 6f 75 72 20 68 6f 73 74 20 6e 61 6d 65 20 61 6e 64 20 65 6d 61 69 6c 20 61 64 64 72 65 73 73 2e 20 49 66 20 79 6f 75 20 64 6f 6e 27 74 20 6c 69 6b 65 20 74 68 69 73 20 70 6f 6c 69 63 79 20 70 6c 65 61 73 65 20 64 69 73 63 6f 6e 6e 65 63 74 20 6e 6f 77 2e 0d 0a 20 20 20 20 50 6c 65 61 73 65 20 62 65 20 61 64 76 69 73 65 64 20 74 68 61 74 20 75 73 65 20 63 6f 6e 73 74 69 74 75 74 65 73 20 63 6f 6e 73 65 6e 74 20 74 6f 20 6d 6f 6e 69 74 6f 72 69 6e 67 20 28 45 6c 65 63 20 43 6f 6d 6d 20 50 72 69 76 20 41 63 74 2c 0d 0a 20 20 20 20 31 38 20 55 53 43 20 32 37 30 31 2d 32 37 31 31 29 2e 20 50 6c 65 61 73 65 20 73 65 65 20 74 68 65 20 66 69 6c 65 20 72 65 61 64 6d 65 2e 74 78 74 20 66 6f 72 20 64 69 73 63 6c 61 69 6d 65 72 73 20 70 65 72 74 61 69 6e 69 6e 67 20 74 6f 20 74 68 69 73 0d 0a 20 20 20 20 73 65 72 76 69 63 65 2e 20 49 66 20 79 6f 75 72 20 46 54 50 20 63 6c 69 65 6e 74 20 63 72 61 73 68 65 73 20 6f 72 20 68 61 6e 67 73 20 73 68 6f 72 74 6c 79 20 61 66 74 65 72 20 6c 6f 67 69 6e 2c 20 74 72 79 20 75 73 69 6e 67 20 61 20 64 61 73 68 0d 0a 20 20 20 20 28 2d 29 20 61 73 20 74 68 65 20 66 69 72 73 74 20 63 68 61 72 61 63 74 65 72 20 6f 66 20 79 6f 75 72 20 70 61 73 73 77 6f 72 64 2e 20 54 68 69 73 20 77 69 6c 6c 20 74 75 72 6e 20 6f 66 66 20 74 68 65 20 69 6e 66 6f 72 6d 61 74 69 6f 6e 61 6c 0d 0a 20 20 20 20 6d 65 73 73 61 67 65 73 20 77 68 69 63 68 20 6d 61 79 20 62 65 20 63 6f 6e 66 75 73 69 6e 67 20 79 6f 75 72 20 66 74 70 20 63 6c 69 65 6e 74 2e 0d 0a 20 20 20 20 2a 2a 2a 2a 2a 2a 2a 2a 49 4e 20 43 41 53 45 20 4f 46 20 50 52 4f 42 4c 45 4d 53 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 0d 0a 20 20 20 20 2a 2a 20 46 69 6c 65 20 43 6f 6e 74 65 6e 74 3a 20 73 65 6e 64 20 45 4d 41 49 4c 20 74 6f 20 64 65 6c 6c 62 62 73 40 64 65 6c 6c 2e 63 6f 6d 20 20 20 2a 2a 0d 0a 20 20 20 20 2a 2a 20 46 54 50 20 53 65 72 76 65 72 3a 20 73 65 6e 64 20 45 4d 41 49 4c 20 74 6f 20 68 6f 73 74 6d 61 73 74 65 72 40 64 65 6c 6c 2e 63 6f 6d 20 20 2a 2a 0d 0a 20 20 20 20 2a 2a 20 57 57 57 20 53 65 72 76 65 72 3a 20 73 65 6e 64 20 45 4d 41 49 4c 20 74 6f 20 77 65 62 6d 61 73 74 65 72 40 64 65 6c 6c 2e 63 6f 6d 20 20 20 2a 2a 0d 0a 20 20 20 20 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 0d 0a
1274    35       0x0400000000004001  1258594163.247187000  0.000009000    0.078494016    0.238593000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           75      0       41         21         21     0x00   64426  1         0x4000  239    0x334e       0x334e       0x2a2a       0x2a2a       0x0040   0                       0                     0                 1050  31    1050    950      0        950     0       1049        30          1070     971       0x0811    0x0018    0x0000      4170    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     32 33 30 20 55 73 65 72 20 6c 6f 67 67 65 64 20 69 6e 2e 0d 0a
1275    35       0x0400000000004000  1258594163.247637000  0.078944000    0.000449984    0.319295000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16253  9         0x4000  128    0x5e91       0x5e91       0x5b5b       0x5b5b       0x0040   0                       0                     0                 31    1071  31      14       971      14      971     30          1070        30       0         0x0011    0x0010    0x0000      63790   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01
1276    35       0x0400000000004000  1258594163.249385000  0.001748000    0.002197984    0.321043000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           66      0       28         8          8      0x00   16254  1         0x4000  128    0x5e88       0x5e88       0x8959       0x8959       0x0040   0                       0                     0                 31    1071  31      0        0        0       0       30          1070        38       8         0x0811    0x0018    0x0000      63790   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     54 59 50 45 20 49 0d 0a
1277    35       0x0400000000004001  1258594163.327121000  0.079934000    0.077736000    0.318527000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           74      0       40         20         20     0x00   1622   -62804    0x4000  239    0x28a4       0x28a4       0xb130       0xb130       0x0044   0                       0                     0                 1071  39    1071    21       8        21      8       1070        38          1090     20        0x0811    0x0018    0x0000      4178    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     32 30 30 20 54 79 70 65 20 73 65 74 20 74 6f 20 49 2e 0d 0a
1278    35       0x0400000000004000  1258594163.327845000  0.078460000    0.000724000    0.399503000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       26         6          6      0x00   16255  1         0x4000  128    0x5e89       0x5e89       0xaaa3       0xaaa3       0x0040   0                       0                     0                 39    1091  39      8        20       8       20      38          1090        44       6         0x0011    0x0018    0x0000      63770   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x01     50 41 53 56 0d 0a
1279    35       0x0400000000004001  1258594163.407582000  0.080461000    0.079737024    0.398988000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           104     0       70         50         50     0x00   5259   3637      0x4000  239    0x1a51       0x1a51       0xbf53       0xbf53       0x0040   0                       0                     0                 1091  45    1091    20       6        20      6       1090        44          1140     50        0x0811    0x0018    0x0000      4184    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     32 32 37 20 45 6e 74 65 72 69 6e 67 20 50 61 73 73 69 76 65 20 4d 6f 64 65 20 28 31 34 33 2c 31 36 36 2c 31 31 2c 31 30 2c 32 35 31 2c 37 38 29 0d 0a
1283    35       0x0400000000004000  1258594163.487490000  0.159645000    0.079907968    0.559148000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           82      0       44         24         24     0x00   16267  12        0x4000  128    0x5e6b       0x5e6b       0xf13a       0xf13a       0x0040   0                       0                     0                 45    1141  45      6        50       6       50      44          1140        68       24        0x0011    0x0018    0x0000      63720   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     53 49 5a 45 20 2f 76 69 64 65 6f 2f 52 37 39 37 33 33 2e 45 58 45 0d 0a
1284    35       0x0400000000004001  1258594163.565990000  0.158408000    0.078500032    0.557396000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           67      0       33         13         13     0x00   11024  5765      0x4000  239    0x03f1       0x03f1       0x049e       0x049e       0x0040   0                       0                     0                 1141  69    1141    50       24       50      24      1140        68          1153     13        0x0811    0x0018    0x0000      4208    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     32 31 33 20 34 32 35 35 30 35 36 0d 0a
1285    35       0x0400000000004000  1258594163.566694000  0.079204000    0.000704000    0.638352000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           82      0       44         24         24     0x00   16268  1         0x4000  128    0x5e6a       0x5e6a       0xf819       0xf819       0x0040   0                       0                     0                 69    1154  69      24       13       24      13      68          1153        92       24        0x0011    0x0018    0x0000      63707   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     52 45 54 52 20 2f 76 69 64 65 6f 2f 52 37 39 37 33 33 2e 45 58 45 0d 0a
1286    35       0x0400000000004001  1258594163.644188000  0.078198000    0.077494016    0.635594000    3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           108     0       74         54         54     0x00   14255  3231      0x4000  239    0xf728       0xf728       0xb8e3       0xb8e3       0x0040   0                       0                     0                 1154  93    1154    13       24       13      24      1153        92          1207     54        0x0811    0x0018    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     31 32 35 20 44 61 74 61 20 63 6f 6e 6e 65 63 74 69 6f 6e 20 61 6c 72 65 61 64 79 20 6f 70 65 6e 3b 20 54 72 61 6e 73 66 65 72 20 73 74 61 72 74 69 6e 67 2e 0d 0a
1303    35       0x0400000000004000  1258594163.838277000  0.271583000    0.194088960    0.909935000    3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   16289  21        0x4000  128    0x5e6d       0x5e6d       0x5b1d       0x5b1d       0x0040   0                       0                     0                 93    1208  93      24       54       24      54      92          1207        92       0         0x0011    0x0010    0x0000      63653   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09
5898    35       0x0400000000004001  1258594185.427515000  21.783327000   21.589238016   22.418921000   3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           78      0       44         24         24     0x00   40815  26560     0x4000  239    0x8f86       0x8f86       0x7425       0x7425       0x0040   0                       0                     0                 1208  93    1208    54       0        54      0       1207        92          1231     24        0x0811    0x0018    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09     32 32 36 20 54 72 61 6e 73 66 65 72 20 63 6f 6d 70 6c 65 74 65 2e 0d 0a
5900    35       0x0400000000004000  1258594185.618346000  21.780069000   0.190830976    22.690004000   3        eth:ipv4:tcp          00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network    49329    143.166.11.10  us       Dell Technologies  21       6        DellInc,US   AppleInc,US  21             ftp           64      0       20         0          0      0x00   18617  2328      0x4000  128    0x5555       0x5555       0x5b1d       0x5b1d       0x0040   0                       0                     0                 93    1232  93      0        24       0       24      92          1231        92       0         0x0011    0x0010    0x0000      63629   0      1460    0       0        0         0x00    0         0x00       0                                                   0x00             0x00                                                        0x09
5902    35       0x0400000000004001  1258594491.683288000  306.255773000  306.064942016  328.674694000  3        eth:ipv4:tcp          00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell Technologies  21       192.168.1.105  07       Private network    49329    6        AppleInc,US  DellInc,US   21             ftp           54      0       20         0          0      0x00   49361  8546      0x4000  239    0x6e3c       0x6e3c       0x431f       0x431f       0x0040   0                       0                     0                 1232  93    1232    24       0        24      0       1231        92          1231     0         0x0811    0x0414    0x0000      4232    0      1380    0       0        0         0x00    0         0x00       0                                                   0x42             0x00                                                        0x09

Try to switch to layer 3 header as homework.

Bit ops on payload

In forensics and protocol reversing, certain bit operations such as bit reversal, nibble flip or shift operations come in handy. Hence, the core of T2 supports all of these ops in packet mode, so that the user is not required to write additional post processing scripts.

tranalyzer2

vi src/main.h

...
// Packet mode (-s option)
#define SPKTMD_PKTNO    1 // Whether or not to print the packet number
#define SPKTMD_PCNTC    1 // Whether or not to print L7 content as characters
#define SPKTMD_PCNTH    0 // Whether or not to print L7 content as hex
#define SPKTMD_PCNTL    4 // 0: Print the full payload of the packet
                          // 1: Print payload from L2
                          // 2: Print payload from L3
                          // 3: Print payload from L4
                          // 4: Print payload from L7

#define SPKTMD_BOPS  0x00 // Operations on content, selected by SPKTMD_PCNTL
                          //    0x00: MSB, no bit inverse, no shift
                          //    0x01: LSB, Bit inverse
                          //    0x02: Nibble SWAP
                          //    0x10: Shift right
                          //    0x20: if 0x10: shift from last byte into extra trailing byte

#define SPKTMD_BSHFT_POS 5 // Bitshift byte pos start
#define SPKTMD_BSHFT     2 // Bitshift
...

SPKTMD_BOPS controls all bit ops on the payload as selected by SPKTMD_PCNTL, so currently on L7 with starting byte position SPKTMD_BSHFT_POS=5. By default no op is selected. The first bit inverts all bits of every byte, the second swaps the nibbles. The 5th shifts all content to the right and the 6th adds a trailing byte where the bit of the last byte will be shifted to. Any of the bits can be selected independently. The sequence of ops is defined as follows if all ops are selected: (0x13)

  1. shift
  2. bit inverse
  3. nibble swap

As homework, try all modes independently. Switch shift from the 10th byte on and look at the hex content, whether you see what is expected. Then try bit inverse, so you can expect bit7->bit0, bit6->bit1, … Nibble swap, should be clear. if you have questions, write the Anteater an email, he’ll help.

Now reset to default output again for the next chapter:

t2conf tranalyzer2 --reset && t2build tranalyzer2

Selecting flows and packets

Maybe you want to look for a certain anomaly or you are interested in all ICMP messages. As our present PCAP does not contain ICMP, download annoloc2.pcap and run t2 on it:

t2 -s -r ~/data/annoloc2.pcap -w ~/results

================================================================================
Tranalyzer 0.9.1 (Anteater), Cobra. PID: 19181, SID: 666
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.9.1
    02: macRecorder, 0.9.1
    03: portClassifier, 0.9.1
    04: basicStats, 0.9.1
    05: tcpFlags, 0.9.1
    06: tcpStates, 0.9.1
    07: icmpDecode, 0.9.1
    08: ftpDecode, 0.9.1
    09: txtSink, 0.9.1
[INF] IPv4 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 481577 (481.58 K)
[INF] IPv6 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 41488 (41.49 K)
[INF] macRecorder: 84110 (84.11 K) short org name records loaded
Processing file: /home/user/data/annoloc2.pcap
Link layer type: Ethernet [EN10MB/1]
Snapshot length: 66
Dump start: 1022171701.691172000 sec (Thu 23 May 2002 16:35:01 GMT)
[WRN] snapL2Length: 54 - snapL3Length: 40 - IP length in header: 1500
Dump stop : 1022171726.640398000 sec (Thu 23 May 2002 16:35:26 GMT)
Total dump duration: 24.949226000 sec
Finished processing. Elapsed time: 4.535087491 sec
Finished unloading flow memory. Time: 4.737622826 sec
Percentage completed: 100.00%
Number of processed packets: 1219015 (1.22 M)
Number of processed bytes: 64082726 (64.08 M)
Number of raw bytes: 844642686 (844.64 M)
Number of pad bytes: 8591685635 (8.59 G)
Number of pcap bytes: 83586990 (83.59 M)
Number of IPv4 packets: 1218588 (1.22 M) [99.96%]
Number of IPv6 packets: 180 [0.01%]
Number of A packets: 561592 (561.59 K) [46.07%]
Number of B packets: 657423 (657.42 K) [53.93%]
Number of A bytes: 29274120 (29.27 M) [45.68%]
Number of B bytes: 34808606 (34.81 M) [54.32%]
<A packet load>: 52.13
<B packet load>: 52.95
--------------------------------------------------------------------------------
macRecorder: MAC pairs per flow: min: 1, max: 2, average: 1.00
basicStats: Flow max(pktload): 1480 (1.48 K)
basicStats: Flow max(b/s), pkts: 19015999488 (19.02 Gb/s), 2
basicStats: Biggest L2 flow talker: 00:d0:02:6d:78:00: 57 [0.00%] packets
basicStats: Biggest L2 flow talker: 00:d0:02:6d:78:00: 2622 (2.62 K) [0.00%] bytes
basicStats: Biggest L3 flow talker: 138.212.189.38 (JP): 23601 (23.60 K) [1.94%] packets
basicStats: Biggest L3 flow talker: 138.212.189.38 (JP): 33731054 (33.73 M) [52.64%] bytes
tcpFlags: Aggregated ipFlags=0x7964
tcpFlags: Aggregated tcpFStat=0x5fff
tcpFlags: Aggregated tcpFlags=0x0fdf
tcpFlags: Aggregated tcpAnomaly=0x33ff
tcpFlags: Aggregated ipToS=0xff
tcpFlags: Number of TCP scans attempted, successful: 959, 886 [92.39%]
tcpFlags: Number of TCP SYN retries, seq retries: 147, 5252 (5.25 K)
tcpFlags: Number WinSz below 1: 1443 (1.44 K) [0.15%]
tcpStates: Aggregated tcpStatesAFlags=0xdf
icmpDecode: Aggregated icmpStat=0x21
icmpDecode: Number of ICMP echo request packets: 224 [7.32%]
icmpDecode: Number of ICMP echo reply packets: 191 [6.24%]
icmpDecode: ICMP echo reply / request ratio: 0.85
ftpDecode: Aggregated ftpStat=0x01
ftpDecode: Number of FTP control packets: 2082 (2.08 K) [0.17%]
--------------------------------------------------------------------------------
Headers count: min: 2, max: 5, average: 3.01
Number of ARP packets: 247 [0.02%]
Number of GRE packets: 20 [0.00%]
Number of IGMP packets: 12 [0.00%]
Number of ICMP packets: 3059 (3.06 K) [0.25%]
Number of ICMPv6 packets: 11 [0.00%]
Number of TCP packets: 948743 (948.74 K) [77.83%]
Number of TCP bytes: 52643546 (52.64 M) [82.15%]
Number of UDP packets: 266900 (266.90 K) [21.89%]
Number of UDP bytes: 11234272 (11.23 M) [17.53%]
Number of IPv4 fragmented packets: 2284 (2.28 K) [0.19%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed      flows: 17603 (17.60 K)
Number of processed L2   flows: 99 [0.56%]
Number of processed IPv4 flows: 17440 (17.44 K) [99.07%]
Number of processed IPv6 flows: 64 [0.36%]
Number of processed A    flows: 9995 (9.99 K) [56.78%]
Number of processed B    flows: 7608 (7.61 K) [43.22%]
Number of request        flows: 9467 (9.47 K) [53.78%]
Number of reply          flows: 8136 (8.14 K) [46.22%]
Total   A/B    flow asymmetry: 0.14
Total req/rply flow asymmetry: 0.08
Number of processed   packets/flows: 69.25
Number of processed A packets/flows: 56.19
Number of processed B packets/flows: 86.41
Number of processed total packets/s: 48859.83 (48.86 K)
Number of processed A+B   packets/s: 48859.83 (48.86 K)
Number of processed A     packets/s: 22509.40 (22.51 K)
Number of processed   B   packets/s: 26350.44 (26.35 K)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<Number of processed flows/s>: 705.55
<Bandwidth>: 270268480 b/s (270.27 Mb/s)
<Snapped bandwidth>: 20548205 b/s (20.55 Mb/s)
<Raw bandwidth>: 270835716 b/s (270.84 Mb/s)
Max number of flows in memory: 15220 (15.22 K) [5.81%]
Memory usage: 0.11 GB [0.17%]
Aggregated flowStat=0x0c0098fa0222d044
[WRN] L3 SnapLength < Length in IP header
[WRN] L4 header snapped
[WRN] Consecutive duplicate IP ID
[WRN] IPv4/6 payload length > framing length
[WRN] IPv4/6 fragmentation header packet missing
[WRN] IPv4/6 packet fragmentation sequence not finished
[INF] Stop dissecting: Clipped packet, unhandled protocol or subsequent fragment
[INF] Layer 2 flows
[INF] IPv4 flows
[INF] IPv6 flows
[INF] ARP
[INF] IPv4/6 fragmentation
[INF] IPv4/6 in IPv4/6
[INF] GRE encapsulation
[INF] GTP tunnel
[INF] SSDP/UPnP

Oups, snap length warning up to the IP header. That’s bad, so we will not see much content, as you can see in the packet file.

tawk 'icmp()' ~/results/annoloc2_flows.txt | head | tcol

%dir  flowInd  flowStat            timeFirst             timeLast              duration     numHdrDesc  numHdrs  hdrDesc        srcMac             dstMac             ethType  vlanID  srcIP           srcIPCC  srcIPOrg                        srcPort  dstIP            dstIPCC  dstIPOrg                          dstPort  l4Proto  macStat  macPairs  srcMac_dstMac_numP                     srcMacLbl_dstMacLbl        dstPortClassN  dstPortClass  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT  aveIAT  stdIAT  pktps  bytps  pktAsm  bytAsm  tcpFStat  ipMindIPID  ipMaxdIPID  ipMinTTL  ipMaxTTL  ipTTLChg  ipToS  ipFlags  ipOptCnt  ipOptCpCl_Num    ip6OptCntHH_D  ip6OptHH_D             tcpISeqN  tcpPSeqCnt  tcpSeqSntBytes  tcpSeqFaultCnt  tcpPAckCnt  tcpFlwLssAckRcvdBytes  tcpAckFaultCnt  tcpBFlgtMx  tcpInitWinSz  tcpAveWinSz  tcpMinWinSz  tcpMaxWinSz  tcpWinSzDwnCnt  tcpWinSzUpCnt  tcpWinSzChgDirCnt  tcpWinSzThRt  tcpFlags  tcpAnomaly  tcpOptPktCnt  tcpOptCnt  tcpOptions  tcpMSS  tcpWS  tcpMPTBF  tcpMPF  tcpMPAID  tcpMPDSSF  tcpTmS  tcpTmER  tcpEcI  tcpUtm    tcpBtm       tcpSSASAATrip  tcpRTTAckTripMin  tcpRTTAckTripMax  tcpRTTAckTripAve  tcpRTTAckTripJitAve  tcpRTTSseqAA  tcpRTTAckJitAve  tcpStatesAFlags  icmpStat  icmpTCcnt  icmpBFTypH_TypL_Code          icmpTmGtw   icmpEchoSuccRatio  icmpPFindex  ftpStat  ftpCDFindex  ftpCC  ftpRC  ftpNumUser  ftpUser  ftpNumPass  ftpPass  ftpNumCP  ftpCP
A     59       0x0400000200004001  1022171701.692762000  1022171701.692762000  0.000000000  1           3        eth:ipv4:icmp  00:80:48:b3:22:ef  00:d0:02:6d:78:00  0x0800           138.212.187.10  jp       "ASAHI KASEI CORPORATION"       0        201.116.148.149  mx       "Uninet SA de CV"                 0        1        0x00     1         00:80:48:b3:22:ef_00:d0:02:6d:78:00_1  COMPEXINC,US_DITECHCOR,US  0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           128       128       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0008  0x00000000  0                  59           0x00                                0                    0                    0
A     893      0x0400000200004001  1022171701.812425000  1022171701.812425000  0.000000000  1           3        eth:ipv4:icmp  00:80:48:d7:ed:7a  00:d0:02:6d:78:00  0x0800           138.212.189.88  jp       "ASAHI KASEI CORPORATION"       0        201.116.161.83   mx       "Uninet SA de CV"                 0        1        0x00     1         00:80:48:d7:ed:7a_00:d0:02:6d:78:00_1  COMPEXINC,US_DITECHCOR,US  0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           128       128       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0008  0x00000000  0                  893          0x00                                0                    0                    0
A     1069     0x0400000200004001  1022171701.889357000  1022171701.889357000  0.000000000  1           3        eth:ipv4:icmp  00:48:54:7a:04:0f  00:d0:02:6d:78:00  0x0800           138.212.184.71  jp       "ASAHI KASEI CORPORATION"       0        146.208.9.41     us       "Keysight Technologies"           0        1        0x00     1         00:48:54:7a:04:0f_00:d0:02:6d:78:00_1  -_DITECHCOR,US             0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           128       128       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0008  0x00000000  0                  1069         0x00                                0                    0                    0
A     1177     0x0400000200004001  1022171701.956543000  1022171701.956543000  0.000000000  1           3        eth:ipv4:icmp  00:d0:02:6d:78:00  00:80:48:b3:22:c4  0x0800           201.118.86.105  mx       "Uninet SA de CV"               0        138.212.189.66   jp       "ASAHI KASEI CORPORATION"         0        1        0x00     1         00:d0:02:6d:78:00_00:80:48:b3:22:c4_1  DITECHCOR,US_COMPEXINC,US  0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           246       246       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0002  0x00000000  0                  1177         0x00                                0                    0                    0
A     1204     0x0400000200004001  1022171701.980834000  1022171701.980834000  0.000000000  1           3        eth:ipv4:icmp  00:d0:02:6d:78:00  00:80:48:b3:22:c4  0x0800           138.213.40.91   br       "Early registration addresses"  0        138.212.189.66   jp       "ASAHI KASEI CORPORATION"         0        1        0x00     1         00:d0:02:6d:78:00_00:80:48:b3:22:c4_1  DITECHCOR,US_COMPEXINC,US  0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           113       113       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0008  0x00000000  0                  1204         0x00                                0                    0                    0
A     1232     0x0400000200004001  1022171702.009674000  1022171702.009674000  0.000000000  1           3        eth:ipv4:icmp  00:48:54:7a:04:0f  00:d0:02:6d:78:00  0x0800           138.212.184.71  jp       "ASAHI KASEI CORPORATION"       0        36.237.77.156    tw       "Data Communication Business Gr"  0        1        0x00     1         00:48:54:7a:04:0f_00:d0:02:6d:78:00_1  -_DITECHCOR,US             0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           128       128       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0008  0x00000000  0                  1232         0x00                                0                    0                    0
A     1557     0x0400000200004001  1022171702.247453000  1022171702.247453000  0.000000000  1           3        eth:ipv4:icmp  00:04:76:22:07:90  00:d0:02:6d:78:00  0x0800           138.212.186.88  jp       "ASAHI KASEI CORPORATION"       0        201.19.77.72     br       "Telemar Norte Leste SA"          0        1        0x00     1         00:04:76:22:07:90_00:d0:02:6d:78:00_1  3com,US_DITECHCOR,US       0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           128       128       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0008  0x00000000  0                  1557         0x00                                0                    0                    0
A     1572     0x0400000200004001  1022171702.265015000  1022171702.265015000  0.000000000  1           3        eth:ipv4:icmp  00:08:a1:1d:3f:f1  00:d0:02:6d:78:00  0x0800           138.212.191.25  jp       "ASAHI KASEI CORPORATION"       0        19.50.144.156    us       "MAINT-APNIC-AP"                  0        1        0x00     1         00:08:a1:1d:3f:f1_00:d0:02:6d:78:00_1  CNetTechI,TW_DITECHCOR,US  0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           128       128       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0008  0x00000000  0                  1572         0x00                                0                    0                    0
A     1718     0x0400000200004001  1022171702.396273000  1022171702.396273000  0.000000000  1           3        eth:ipv4:icmp  00:80:48:b3:24:eb  00:d0:02:6d:78:00  0x0800           138.212.190.25  jp       "ASAHI KASEI CORPORATION"       0        19.6.20.159      us       "MAINT-APNIC-AP"                  0        1        0x00     1         00:80:48:b3:24:eb_00:d0:02:6d:78:00_1  COMPEXINC,US_DITECHCOR,US  0              unknown       1           0            28           0             28        28        28          28          0       0       0       0       0      0      -1      1       0x0001    65535       0           128       128       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0         0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      1          0x00000000_0x00000008_0x0008  0x00000000  0                  1718         0x00                                0                    0                    0

icmpDecode links an ICMP message to the flow who caused it, so with a one liner we can select all the linked flows from the flow file.

tawk -H 'icmp() && $icmpPFindex { printf "%d;", $icmpPFindex } END { printf "\n" }' ~/results/annoloc2_flows.txt

59;893;1069;1177;1204;1232;1557;1572;1718;1741;1750;1820;1877;1877;1906;1987;1996;2037;2092;2124;2228;2253;2389;2469;2504;2523;2532;2540;2571;2588;2655;2692;2725;2745;1161;2835;2847;2899;2921;2927;2943;2971;2972;3002;3024;3107;3128;3131;3152;3165;3229;398;3244;3348;2385;1657;1549;1746;2362;1801;3590;1910;1807;3612;1896;591;2099;2137;3707;1482;2176;2180;3815;2770;2505;2539;3251;2850;2763;2942;2779;3047;2695;4325;3113;3157;3149;3172;3212;2649;4408;1137;637;1055;1226;1217;3267;4516;4561;3499;1593;4601;1652;4611;1701;4613;1922;2330;4633;1774;1929;1854;2558;2042;2570;3659;2091;4871;2665;2486;2466;4938;2882;2596;2769;2580;3001;5079;2759;2716;2908;1465;3008;5233;3163;3234;4525;3188;3261;3603;5477;3328;5503;5532;3464;3701;2121;3416;3468;3508;2906;5642;3548;4688;4736;3720;5827;3698;5901;4949;3872;3572;3998;4120;3273;4130;5237;4221;4261;3235;4297;5335;4165;4357;4377;6322;4299;3718;5434;4416;6425;4500;4764;4509;5811;4664;6520;4695;4717;4763;6628;4823;6607;5525;4828;4864;4691;4998;6775;4977;5708;6816;6830;4941;4969;5135;4968;5403;5124;4556;5316;5249;5248;5252;5272;4669;6397;7188;5360;5422;5494;5390;5367;5581;5651;5636;5630;5589;6595;5012;5501;5659;7433;5666;5731;5698;3434;5750;6915;1436;7063;6684;5887;6859;6772;5877;6945;6946;5970;6881;5822;5934;5988;5347;5419;6041;5430;6063;5125;6065;7299;6260;6089;6140;7046;6232;6200;6179;6300;7578;6339;6380;6362;6390;5829;6440;6370;6536;6509;6828;6574;8350;6650;6649;8463;6759;6760;6755;6740;6783;6799;6834;8060;5223;6879;1684;6892;6932;6908;6889;6997;7037;8897;8917;7974;7222;7102;7416;7211;9015;8157;7298;8077;1531;2852;8230;7463;7511;9168;8140;7338;1338;1308;3928;977;9215;7454;9235;1935;7439;1157;1947;9263;1972;2052;8313;9292;8365;8351;8708;8083;1358;6972;1587;2086;1489;2261;9333;9340;7886;8427;9353;4020;1933;8421;9360;8451;1624;2217;7527;7667;9378;9385;7548;9402;8545;7609;784;8508;7615;9422;2347;8453;9441;7645;9465;2205;9487;9498;9500;8285;7668;8560;9535;1764;2335;2358;9523;7688;8619;7705;2103;7722;2604;7752;8320;9607;8108;2232;7753;9627;9629;8769;2263;9641;8006;7699;866;2375;9662;9666;3330;9673;8729;9680;2633;7836;9701;9711;8467;3516;9067;9726;8843;7848;7784;2734;9180;8051;8809;2761;7907;8009;8433;8816;2356;2813;7921;2855;6249;2840;7913;1144;8583;9792;2669;2521;1870;9834;8906;9843;3142;2980;380;8043;9869;3028;9875;790;8923;9889;9891;8091;2965;9925;3144;1013;3201;575;3207;730;8966;3037;731;9976;1029;540;1051;8158;9415;1111;

And select some of them:

tawk 'flow("889;1051;1165;1179;1221;1554")' ~/results/annoloc2_flows.txt | tcol

%dir  flowInd  flowStat            timeFirst             timeLast              duration      numHdrDesc  numHdrs  hdrDesc        srcMac             dstMac             ethType  vlanID  srcIP            srcIPCC  srcIPOrg                   srcPort  dstIP            dstIPCC  dstIPOrg                   dstPort  l4Proto  macStat  macPairs  srcMac_dstMac_numP                       srcMacLbl_dstMacLbl        dstPortClassN  dstPortClass  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT    aveIAT     stdIAT     pktps      bytps     pktAsm       bytAsm      tcpFStat  ipMindIPID  ipMaxdIPID  ipMinTTL  ipMaxTTL  ipTTLChg  ipToS  ipFlags  ipOptCnt  ipOptCpCl_Num    ip6OptCntHH_D  ip6OptHH_D             tcpISeqN    tcpPSeqCnt  tcpSeqSntBytes  tcpSeqFaultCnt  tcpPAckCnt  tcpFlwLssAckRcvdBytes  tcpAckFaultCnt  tcpBFlgtMx  tcpInitWinSz  tcpAveWinSz  tcpMinWinSz  tcpMaxWinSz  tcpWinSzDwnCnt  tcpWinSzUpCnt  tcpWinSzChgDirCnt  tcpWinSzThRt  tcpFlags  tcpAnomaly  tcpOptPktCnt  tcpOptCnt  tcpOptions  tcpMSS  tcpWS  tcpMPTBF  tcpMPF  tcpMPAID  tcpMPDSSF  tcpTmS  tcpTmER  tcpEcI  tcpUtm    tcpBtm       tcpSSASAATrip  tcpRTTAckTripMin  tcpRTTAckTripMax  tcpRTTAckTripAve  tcpRTTAckTripJitAve  tcpRTTSseqAA  tcpRTTAckJitAve  tcpStatesAFlags  icmpStat  icmpTCcnt  icmpBFTypH_TypL_Code          icmpTmGtw   icmpEchoSuccRatio  icmpPFindex  ftpStat  ftpCDFindex  ftpCC  ftpRC  ftpNumUser  ftpUser  ftpNumPass  ftpPass  ftpNumCP  ftpCP
A     889      0x0400000200004000  1022171701.811421000  1022171702.089237000  0.277816000   1           3        eth:ipv4:udp   00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800           170.197.107.104  us       "Sea-Land Services"        18164    138.212.185.98   jp       "ASAHI KASEI CORPORATION"  28015    17       0x00     1         00:d0:02:6d:78:00_00:00:e8:8f:b7:93_2    DITECHCOR,US_AcctonTec,US  28015          unknown       2           2            17           1192          8         9         8.5         6.403124    0       0.277816  0.138908   0.1964456  7.199009   61.19158  0            -0.9718776  0x0001    25          25          107       107       0         0x00   0x3800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0           0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0.06713402     0.037482          0.06713402        0.05230801        0.02096714           0.277816      0                0x00             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
B     889      0x0400000200004001  1022171701.878555000  1022171702.126719000  0.248164000   1           3        eth:ipv4:udp   00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800           138.212.185.98   jp       "ASAHI KASEI CORPORATION"  28015    170.197.107.104  us       "Sea-Land Services"        18164    17       0x00     1         00:00:e8:8f:b7:93_00:d0:02:6d:78:00_2    AcctonTec,US_DITECHCOR,US  28015          unknown       2           2            1192         17            98        1094      596         1046.148    0       0.248164  0.124082   0.1754784  8.059187   4803.275  0            0.9718776   0x0001    100         100         64        64        0         0x00   0x3800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0           0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0.210682       0.210682          0.210682          0.210682          0                    0.26299       0.02096714       0x00             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
A     1554     0x0400000200004000  1022171702.246761000  1022171702.246761000  0.000000000   1           3        eth:ipv4:udp   00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800           193.175.162.199  de       "DFN-LIR-MNT"              1138     138.212.185.98   jp       "ASAHI KASEI CORPORATION"  27015    17       0x00     1         00:d0:02:6d:78:00_00:00:e8:8f:b7:93_1    DITECHCOR,US_AcctonTec,US  27015          unknown       1           1            9            6             9         9         9           9           0       0         0          0          0          0         0            0.2         0x0001    65535       0           117       117       0         0x00   0x3800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0           0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0.008453008    0.008453008       0.008453008       0.008453008       0                    0.008453008   0                0x00             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
B     1554     0x0400000200004001  1022171702.255214000  1022171702.255214000  0.000000000   1           3        eth:ipv4:udp   00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800           138.212.185.98   jp       "ASAHI KASEI CORPORATION"  27015    193.175.162.199  de       "DFN-LIR-MNT"              1138     17       0x00     1         00:00:e8:8f:b7:93_00:d0:02:6d:78:00_1    AcctonTec,US_DITECHCOR,US  27015          unknown       1           1            6            9             6         6         6           6           0       0         0          0          0          0         0            -0.2        0x0001    65535       0           64        64        0         0x00   0x3800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0           0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0.008453008   0                0x00             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
A     1221     0x0400000200004000  1022171702.003105000  1022171726.232882000  24.229777000  1           3        eth:ipv4:tcp   00:d0:02:6d:78:00  00:00:1c:b6:15:75  0x0800           173.143.243.87   us       "Sprint Communications"    1313     138.212.184.34   jp       "ASAHI KASEI CORPORATION"  4662     6        0x00     1         00:d0:02:6d:78:00_00:00:1c:b6:15:75_74   DITECHCOR,US_BellTech,US   4662           oms           74          107          46           130055        0         41        0.6216216   4.806583    0       2.479512  0.3274294  0.414261   3.054093   1.89849   -0.1823204   -0.9992929  0x0811    1281        22528       111       111       0         0x00   0x1844   0         0x00_0x00000000  0_0            0x00000000_0x00000000  43342326    74          46              0               74          133424                 1               41          8484          8406.719     7195         8484         11              11             11                 0             0x0058    0x0100      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              0.002586048       1.825197          0.03973731        0.1775885            0             0                0x03             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
B     1221     0x0400000200004001  1022171702.009685000  1022171725.837036000  23.827351000  1           3        eth:ipv4:tcp   00:00:1c:b6:15:75  00:d0:02:6d:78:00  0x0800           138.212.184.34   jp       "ASAHI KASEI CORPORATION"  4662     173.143.243.87   us       "Sprint Communications"    1313     6        0x00     1         00:00:1c:b6:15:75_00:d0:02:6d:78:00_107  BellTech,US_DITECHCOR,US   4662           oms           107         74           130055       46            0         1414      1215.467    477.4761    0       2.658446  0.2226855  0.4524964  4.490638   5458.223  0.1823204    0.9992929   0x0011    1           2048        128       128       0         0x00   0x1844   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1686136000  106         130596          0               107         46                     0               7075        63344         63298        63298        63344        2               0              0                  0             0x0058    0x0200      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              0.000567008       1.668696          0.4638892         0.400939             0.5036265     0.4385085        0x03             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
A     1179     0x0400000200004000  1022171701.958774000  1022171726.616651000  24.657877000  1           3        eth:ipv4:tcp   00:d0:02:6d:78:00  00:80:48:b3:22:c4  0x0800           83.24.78.221     pl       "TPNET"                    3204     138.212.189.66   jp       "ASAHI KASEI CORPORATION"  1214     6        0x00     1         00:d0:02:6d:78:00_00:80:48:b3:22:c4_33   DITECHCOR,US_COMPEXINC,US  1214           kazaa         33          32           1975         1275          0         1460      59.84848    254.1091    0       4.173618  0.7472084  0.8226424  1.338315   80.09611  0.01538462   0.2153846   0x0811    1           198         115       115       0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3045997063  33          1918            0               33          1275                   0               1460        16070         16401.18     16070        17520        21              1              22                 0             0x0058    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              1.9008e-05        1.458637          0.3520578         0.4273103            0             0                0x03             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
B     1179     0x0400000200004001  1022171702.148305000  1022171726.177771000  24.029466000  1           3        eth:ipv4:tcp   00:80:48:b3:22:c4  00:d0:02:6d:78:00  0x0800           138.212.189.66   jp       "ASAHI KASEI CORPORATION"  1214     83.24.78.221     pl       "TPNET"                    3204     6        0x00     1         00:80:48:b3:22:c4_00:d0:02:6d:78:00_32   COMPEXINC,US_DITECHCOR,US  1214           kazaa         32          33           1275         1975          0         101       39.84375    31.50902    0       4.546829  0.7509208  0.8038685  1.331698   53.05985  -0.01538462  -0.2153846  0x0811    31          789         128       128       0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1389674681  32          1174            0               32          1861                   0               101         63937         64119.21     63645        64240        10              1              4                  0             0x0058    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              0.003298          4.381874          0.4854233         0.7526243            0.8374811     0.8654694        0x03             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
A     1051     0x0400000200004001  1022171701.880886000  1022171726.605036000  24.724150000  1           3        eth:ipv4:icmp  00:d0:02:6d:78:00  00:04:75:73:9b:a2  0x0800           19.6.23.127      us       "MAINT-APNIC-AP"           0        138.212.191.240  jp       "ASAHI KASEI CORPORATION"  0        1        0x00     1         00:d0:02:6d:78:00_00:04:75:73:9b:a2_88   DITECHCOR,US_3com,US       0              unknown       88          0            2464         0             28        28        28          2.98481     0       0.489347  0.2809563  0.1034217  3.559273   99.65965  -1           1           0x0001    9           46          116       116       0         0x00   0x1800   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0           0           0               0               0           0                      0               0           0             0            0            0            0               0              0                  0             0x0000    0x0000      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              65535             0                 0                 0                    0             0                0x00             0x01      88         0x00000000_0x00000008_0x0008  0x00000000  0                  1051         0x00                                0                    0                    0
A     1165     0x0400000200004000  1022171701.947098000  1022171726.635380000  24.688282000  1           3        eth:ipv4:tcp   00:d0:02:6d:78:00  00:80:48:b3:22:c4  0x0800           133.151.116.195  jp       "Brother Industries"       1857     138.212.189.66   jp       "ASAHI KASEI CORPORATION"  1214     6        0x00     1         00:d0:02:6d:78:00_00:80:48:b3:22:c4_11   DITECHCOR,US_COMPEXINC,US  1214           kazaa         11          12           3384         1534          0         1460      307.6364    557.4133    0       6.342256  2.244389   2.19089    0.4455555  137.0691  -0.04347826  0.3761692   0x0811    1           261         121       121       0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  4188389891  10          3384            0               11          1222                   0               1460        16943         17241.4      16202        17520        7               1              4                  0             0x0018    0x0040      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              2.8032e-05        3.717183          0.4699067         1.33743              0             0                0x03             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0
B     1165     0x0400000200004001  1022171701.947591000  1022171726.635505000  24.687914000  1           3        eth:ipv4:tcp   00:80:48:b3:22:c4  00:d0:02:6d:78:00  0x0800           138.212.189.66   jp       "ASAHI KASEI CORPORATION"  1214     133.151.116.195  jp       "Brother Industries"       1857     6        0x00     1         00:80:48:b3:22:c4_00:d0:02:6d:78:00_12   COMPEXINC,US_DITECHCOR,US  1214           kazaa         12          11           1534         3384          0         271       127.8333    111.9265    0       3.71669   2.057326   1.41717    0.4860678  62.13567  0.04347826   -0.3761692  0x0811    50          723         128       128       0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1092976442  11          1222            0               12          3384                   0               328         64240         63558.79     63053        64240        2               1              2                  0             0x0018    0x0040      0             0          0x00000000  0       1      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000000  0              0.08887299        2.625073          1.73182           1.165039             2.201727      1.773707         0x03             0x00      0          0x00000000_0x00000000_0x0000  0x00000000  0                  0            0x00                                0                    0                    0

Unfortunately there is no content in the pcap, otherwise you could see the content of the packet listing the ICMP packets going back to the sender.

If you looked closely at flowStat in the end report or the snap length warning:


[WRN] snapL2Length: 54 - snapL3Length: 40 - IP length in header: 1500

From the L3 header 40 bytes are left.

Decode the flowStat column:

tawk -V flowStat=0x0400000200004000


The flowStat column with value 0x0400000200004000 is to be interpreted as follows:

   bit | flowStat              | Description
   =============================================================================
    14 | 0x0000 0000 0000 4000 | IPv4 flow
    33 | 0x0000 0002 0000 0000 | Acquired packet length < packet length in L3 header
    58 | 0x0400 0000 0000 0000 | IPv4 packet

So we do not see any L7 content in the packet file, because it got snapped.

Why would somebody do that? Right, anonymization…

Wireshark like follow stream

In this section, we will analyze faf-exercise.pcap again. We will see how we can use tawk follow_stream() function to reconstruct the payload of a given flow (akin to Wireshark Follow TCP Stream and Follow UDP Stream functionality).

Let’s see how it can be used:

t2 -s -r ~/data/faf-exercise.pcap -w ~/results

================================================================================
Tranalyzer 0.9.1 (Anteater), Cobra. PID: 19263, SID: 666
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.9.1
    02: macRecorder, 0.9.1
    03: portClassifier, 0.9.1
    04: basicStats, 0.9.1
    05: tcpFlags, 0.9.1
    06: tcpStates, 0.9.1
    07: icmpDecode, 0.9.1
    08: ftpDecode, 0.9.1
    09: txtSink, 0.9.1
[INF] IPv4 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 481577 (481.58 K)
[INF] IPv6 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 41488 (41.49 K)
[INF] macRecorder: 84110 (84.11 K) short org name records loaded
Processing file: /home/user/data/faf-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
...
tawk -d follow_stream

follow_stream(f, of, d, pf, r, nc):

  Return the payload of flow with index 'f'.

  Parameters:
    - f: flow index to follow.
    - [of]: output format [default: 0]:
            0: Payload only,
            1: Prefix each payload with packet/flow info,
            2: JSON,
            3: Reconstruct (pipe the output to 'xxd -p -r' to reproduce the binary file).
    - [d]: direction to follow ("A" or "B") [default: "" (A and B)]
    - [pf]: payload format [default: 0]:
            0: ASCII,
            1: Hexdump,
            2: Raw/Binary,
            3: Base64.
    - [r]: do not analyze TCP sequence numbers (no TCP reassembly and reordering) [default: 0]
    - [nc]: do not output colors [default: 0]

  Dependencies:
    - basicFlow
    - basicStats
    - tcpFlags

  Examples:
    # follow stream 1 with payload as ASCII.
    $ tawk 'follow_stream(1)' file.txt

    # follow stream 2 with packet/flow info and payload as ASCII.
    $ tawk 'follow_stream(2, 1)' file.txt

    # follow stream 3 with packet/flow info as JSON and payload as ASCII.
    $ tawk 'follow_stream(3, 2)' file.txt

    # follow stream 4, direction B only, without packet/flow info and
    # reconstruct payload (original raw data) as binary into 'out.data'.
    $ tawk 'tawk 'follow_stream(4, 3, "B")' file.txt | xxd -p -r > out.data

    # follow stream 5, direction A only, without packet/flow info and payload as hexdump
    $ tawk 'follow_stream(5, 0, "A", 1)' file.txt

    # follow stream 6, both directions as JSON and payload as base64.
    $ tawk 'follow_stream(6, 2, "", 3)' file.txt

    # follow stream 7, both directions with packet/flow info and payload as binary.
    $ tawk 'follow_stream(7, 1, "AB", 2)' file.txt

    # follow stream 8 with payload as ASCII, without packet/flow info, with TCP
    # sequence number analysis and without colors, and redirect output to 'out.txt'.
    $ tawk 'follow_stream(8, 0, "", 0, 0, 1)' file.txt > out.txt

    # follow stream 9 with payload as hexdump, with packet/flow info,
    # but without TCP sequence numbers analysis and colors.
    $ tawk 'follow_stream(9, 1, "", 1, 1, 1)' file.txt

Let’s follow stream number 1:

tawk 'follow_stream(1)' ~/results/faf-exercise_packets.txt

GET /softw/90/update/avg9infoavi.ctf HTTP/1.1
User-Agent: AVGINET9-WXPPX86 90 AVI=270.14.71/2510 BUILD=707 LOC=1033 LIC=9I-ASXNN-X4WGW-M0XFR-T84VX-3VX02 DIAG=51E OPF=0 PCA=
Host: backup.avg.cz
Accept: */*
Accept-Encoding: identity,deflate,gzip
If-Modified-Since: Tue, 17 Nov 2009 20:41:10 GMT
Pragma: no-cache
Cache-Control: no-cache
x-avg-id:78-175947826

HTTP/1.1 200 OK
Date: Wed, 18 Nov 2009 11:39:48 GMT
Server: Apache
Last-Modified: Wed, 18 Nov 2009 09:04:15 GMT
ETag: "15c007-cea-478a186e401c0"
Accept-Ranges: bytes
Content-Length: 3306
Connection: close
Content-Type: text/plain

AVG CTF Index File;ver(10)
bin(u7avi1777ff.bin)grp(avi:1777)tm(0911180855)pri(2)len(6637487)
bin(u7avi1777u1323ff.bin)grp(avi:1777)dep(avi:1323)tm(0911180855)pri(2)len(590030)
bin(u7avi1777u1705ff.bin)grp(avi:1777)dep(avi:1705)tm(0911180855)pri(2)len(95323)
bin(u7iavi2511ff.bin)grp(iavi:2511)tm(0911180855)pri(2)len(45641416)
bin(u7iavi2511u2251fo.bin)grp(iavi:2511)dep(iavi:2251)tm(0911180855)pri(2)len(6175342)
bin(u7iavi2511u2353fn.bin)grp(iavi:2511)dep(iavi:2353)tm(0911180855)pri(2)len(4506824)
bin(u7iavi2511u2404fm.bin)grp(iavi:2511)dep(iavi:2404)tm(0911180855)pri(2)len(3342546)
bin(u7iavi2511u2431fl.bin)grp(iavi:2511)dep(iavi:2431)tm(0911180855)pri(2)len(2609643)
bin(u7iavi2511u2458fk.bin)grp(iavi:2511)dep(iavi:2458)tm(0911180855)pri(2)len(1380674)
bin(u7iavi2511u2480fj.bin)grp(iavi:2511)dep(iavi:2480)tm(0911180855)pri(2)len(681743)
bin(u7iavi2511u2490fi.bin)grp(iavi:2511)dep(iavi:2490)tm(0911180855)pri(2)len(555496)
bin(u7iavi2511u2500fh.bin)grp(iavi:2511)dep(iavi:2500)tm(0911180855)pri(2)len(332906)
bin(u7iavi2511u2505fh.bin)grp(iavi:2511)dep(iavi:2505)tm(0911180855)pri(2)len(204942)
bin(u7iavi2511u2508fg.bin)grp(iavi:2511)dep(iavi:2508)tm(0911180855)pri(2)len(72274)
bin(u7iavi2511u2510ff.bin)grp(iavi:2511)dep(iavi:2510)tm(0911180855)pri(2)len(30540)
bin(u9ichjw4qt.bin)grp(ichjw:4)pri(2)tm(0907131859)len(113488)
bin(u9ichjw4u2ia.bin)grp(ichjw:4)dep(ichjw:2)pri(2)tm(0907131859)len(94470)
bin(u9ichjw4u3gq.bin)grp(ichjw:4)dep(ichjw:3)pri(2)tm(0907131859)len(13313)
bin(u9idat249b243tc.bin)grp(idat:249)dif(idat:243)pri(2)tm(0911051401)len(621430)
bin(u9idat249b245bp.bin)grp(idat:249)dif(idat:245)pri(2)tm(0911051401)len(197648)
bin(u9idat249b246mj.bin)grp(idat:249)dif(idat:246)pri(2)tm(0911051400)len(159289)
bin(u9idat249b247uh.bin)grp(idat:249)dif(idat:247)pri(2)tm(0911051400)len(95680)
bin(u9idat249le.bin)grp(idat:249)pri(2)tm(0911051400)len(1927074)
bin(u9ifw42jx.bin)grp(ifw:42)pri(2)tm(0911131400)len(541853)
bin(u9ifw42u38we.bin)grp(ifw:42)dep(ifw:38)pri(2)tm(0911131400)len(123575)
bin(u9ifw42u39ke.bin)grp(ifw:42)dep(ifw:39)pri(2)tm(0911131400)len(92745)
bin(u9ifw42u40dc.bin)grp(ifw:42)dep(ifw:40)pri(2)tm(0911131400)len(67292)
bin(u9ifw42u41tz.bin)grp(ifw:42)dep(ifw:41)pri(2)tm(0911131400)len(18393)
bin(x8all234yc.bin)grp(xplph:0012;xplsb:0099;xplsb2:0118;xplsc:0149)tm(0911180700)pri(2)len(1146352)
bin(x8xplph_12gj.bin)grp(xplph:12)tm(0910280700)pri(2)len(4551)
bin(x8xplsb2_118c8.bin)grp(xplsb2:118)tm(0911180700)pri(2)len(4989)
bin(x8xplsb_9946.bin)grp(xplsb:99)tm(0911160700)pri(2)len(985460)
bin(x8xplsb_99d9546.bin)grp(xplsb:99)dif(xplsb:95)tm(0911160700)pri(2)len(32267)
bin(x8xplsb_99d9646.bin)grp(xplsb:99)dif(xplsb:96)tm(0911160700)pri(2)len(3505)
bin(x8xplsb_99d9746.bin)grp(xplsb:99)dif(xplsb:97)tm(0911160700)pri(2)len(644)
bin(x8xplsb_99d9846.bin)grp(xplsb:99)dif(xplsb:98)tm(0911160700)pri(2)len(581)
bin(x8xplsc_149c8.bin)grp(xplsc:149)tm(0911180700)pri(2)len(151922)
bin(x8xplsc_149d145c8.bin)grp(xplsc:149)dif(xplsc:145)tm(0911180700)pri(2)len(2561)
bin(x8xplsc_149d146c8.bin)grp(xplsc:149)dif(xplsc:146)tm(0911180700)pri(2)len(1921)
bin(x8xplsc_149d147c8.bin)grp(xplsc:149)dif(xplsc:147)tm(0911180700)pri(2)len(1767)
bin(x8xplsc_149d148c8.bin)grp(xplsc:149)dif(xplsc:148)tm(0911180700)pri(2)len(1411)

If you want to know which data belongs to which packet, you can use the following command instead:

tawk 'follow_stream(1, 1)' ~/results/faf-exercise_packets.txt

================================================================================
Packet 4 (367 bytes): flow 1 A 192.168.1.104:1258 -> 77.67.44.206:80 TCP seq: 3579665154
================================================================================

GET /softw/90/update/avg9infoavi.ctf HTTP/1.1
User-Agent: AVGINET9-WXPPX86 90 AVI=270.14.71/2510 BUILD=707 LOC=1033 LIC=9I-ASXNN-X4WGW-M0XFR-T84VX-3VX02 DIAG=51E OPF=0 PCA=
Host: backup.avg.cz
Accept: */*
Accept-Encoding: identity,deflate,gzip
If-Modified-Since: Tue, 17 Nov 2009 20:41:10 GMT
Pragma: no-cache
Cache-Control: no-cache
x-avg-id:78-175947826



================================================================================
Packet 5 (1380 bytes): flow 1 B 77.67.44.206:80 -> 192.168.1.104:1258 TCP seq: 83075340
================================================================================

HTTP/1.1 200 OK
Date: Wed, 18 Nov 2009 11:39:48 GMT
Server: Apache
Last-Modified: Wed, 18 Nov 2009 09:04:15 GMT
ETag: "15c007-cea-478a186e401c0"
Accept-Ranges: bytes
Content-Length: 3306
Connection: close
Content-Type: text/plain

AVG CTF Index File;ver(10)
bin(u7avi1777ff.bin)grp(avi:1777)tm(0911180855)pri(2)len(6637487)
bin(u7avi1777u1323ff.bin)grp(avi:1777)dep(avi:1323)tm(0911180855)pri(2)len(590030)
bin(u7avi1777u1705ff.bin)grp(avi:1777)dep(avi:1705)tm(0911180855)pri(2)len(95323)
bin(u7iavi2511ff.bin)grp(iavi:2511)tm(0911180855)pri(2)len(45641416)
bin(u7iavi2511u2251fo.bin)grp(iavi:2511)dep(iavi:2251)tm(0911180855)pri(2)len(6175342)
bin(u7iavi2511u2353fn.bin)grp(iavi:2511)dep(iavi:2353)tm(0911180855)pri(2)len(4506824)
bin(u7iavi2511u2404fm.bin)grp(iavi:2511)dep(iavi:2404)tm(0911180855)pri(2)len(3342546)
bin(u7iavi2511u2431fl.bin)grp(iavi:2511)dep(iavi:2431)tm(0911180855)pri(2)len(2609643)
bin(u7iavi2511u2458fk.bin)grp(iavi:2511)dep(iavi:2458)tm(0911180855)pri(2)len(1380674)
bin(u7iavi2511u2480fj.bin)grp(iavi:2511)dep(iavi:2480)tm(0911180855)pri(2)len(681743)
bin(u7iavi2511u2490fi.bin)grp(iavi:2511)dep(iavi:2490)tm(0911180855)pri(2)len(555496)
bin(u7iavi2511u2500fh.bin)grp(iavi:2511)dep(iavi:2500)tm(0911180855)pri(2)len(332906)
bin(u7iavi2511u2505fh.bin)grp(iavi:2511)dep(iavi:2505)tm(0911180855)pri(2)len(204942)
bin(u7iavi2511u2508

================================================================================
Packet 6 (321 bytes): flow 1 B 77.67.44.206:80 -> 192.168.1.104:1258 TCP seq: 83076720
================================================================================

fg.bin)grp(iavi:2511)dep(iavi:2508)tm(0911180855)pri(2)len(72274)
bin(u7iavi2511u2510ff.bin)grp(iavi:2511)dep(iavi:2510)tm(0911180855)pri(2)len(30540)
bin(u9ichjw4qt.bin)grp(ichjw:4)pri(2)tm(0907131859)len(113488)
bin(u9ichjw4u2ia.bin)grp(ichjw:4)dep(ichjw:2)pri(2)tm(0907131859)len(94470)
bin(u9ichjw4u3gq.bin)grp(ic

================================================================================
Packet 8 (1380 bytes): flow 1 B 77.67.44.206:80 -> 192.168.1.104:1258 TCP seq: 83077041
================================================================================

hjw:4)dep(ichjw:3)pri(2)tm(0907131859)len(13313)
bin(u9idat249b243tc.bin)grp(idat:249)dif(idat:243)pri(2)tm(0911051401)len(621430)
bin(u9idat249b245bp.bin)grp(idat:249)dif(idat:245)pri(2)tm(0911051401)len(197648)
bin(u9idat249b246mj.bin)grp(idat:249)dif(idat:246)pri(2)tm(0911051400)len(159289)
bin(u9idat249b247uh.bin)grp(idat:249)dif(idat:247)pri(2)tm(0911051400)len(95680)
bin(u9idat249le.bin)grp(idat:249)pri(2)tm(0911051400)len(1927074)
bin(u9ifw42jx.bin)grp(ifw:42)pri(2)tm(0911131400)len(541853)
bin(u9ifw42u38we.bin)grp(ifw:42)dep(ifw:38)pri(2)tm(0911131400)len(123575)
bin(u9ifw42u39ke.bin)grp(ifw:42)dep(ifw:39)pri(2)tm(0911131400)len(92745)
bin(u9ifw42u40dc.bin)grp(ifw:42)dep(ifw:40)pri(2)tm(0911131400)len(67292)
bin(u9ifw42u41tz.bin)grp(ifw:42)dep(ifw:41)pri(2)tm(0911131400)len(18393)
bin(x8all234yc.bin)grp(xplph:0012;xplsb:0099;xplsb2:0118;xplsc:0149)tm(0911180700)pri(2)len(1146352)
bin(x8xplph_12gj.bin)grp(xplph:12)tm(0910280700)pri(2)len(4551)
bin(x8xplsb2_118c8.bin)grp(xplsb2:118)tm(0911180700)pri(2)len(4989)
bin(x8xplsb_9946.bin)grp(xplsb:99)tm(0911160700)pri(2)len(985460)
bin(x8xplsb_99d9546.bin)grp(xplsb:99)dif(xplsb:95)tm(0911160700)pri(2)len(32267)
bin(x8xplsb_99d9646.bin)grp(xplsb:99)dif(xplsb:96)tm(0911160700)pri(2)len(3505)
bin(x8xplsb_99d9746.bin)grp(xplsb:99)dif(xplsb:97)tm(0911160700)pri(2)len(644)
bin(x8xplsb_99d9846.bin

================================================================================
Packet 9 (466 bytes): flow 1 B 77.67.44.206:80 -> 192.168.1.104:1258 TCP seq: 83078421
================================================================================

)grp(xplsb:99)dif(xplsb:98)tm(0911160700)pri(2)len(581)
bin(x8xplsc_149c8.bin)grp(xplsc:149)tm(0911180700)pri(2)len(151922)
bin(x8xplsc_149d145c8.bin)grp(xplsc:149)dif(xplsc:145)tm(0911180700)pri(2)len(2561)
bin(x8xplsc_149d146c8.bin)grp(xplsc:149)dif(xplsc:146)tm(0911180700)pri(2)len(1921)
bin(x8xplsc_149d147c8.bin)grp(xplsc:149)dif(xplsc:147)tm(0911180700)pri(2)len(1767)
bin(x8xplsc_149d148c8.bin)grp(xplsc:149)dif(xplsc:148)tm(0911180700)pri(2)len(1411)


================================================================================
1 client packets, 4 server packets, 1 turns.

If you want the output as hex?! Easy, let’s configure the core to output the packet content as hex (SPKTMD_PCNTH=1). Note that to improve the execution time, we could also switch the ASCII output off (SPKTMD_PCNTC=0).

t2conf tranalyzer2 -D SPKTMD_PCNTH=1 && t2build tranalyzer2

t2 -r ~/data/faf-exercise.pcap -w ~/results/ -s

Let’s analyze flow 35 for a change and discard the packet info:

tawk 'follow_stream(35, 0, "", 1)' ~/results/faf-exercise_packets.txt

    00000000  32 32 30 20 4d 69 63 72  6f 73 6f 66 74 20 46 54   220 Micr osoft FT
    00000010  50 20 53 65 72 76 69 63  65 0d 0a                  P Servic e..
00000000  55 53 45 52 20 61 6e 6f  6e 79 6d 6f 75 73 0d 0a   USER ano nymous..
    0000001B  33 33 31 20 41 6e 6f 6e  79 6d 6f 75 73 20 61 63   331 Anon ymous ac
    0000002B  63 65 73 73 20 61 6c 6c  6f 77 65 64 2c 20 73 65   cess all owed, se
    0000003B  6e 64 20 69 64 65 6e 74  69 74 79 20 28 65 2d 6d   nd ident ity (e-m
    0000004B  61 69 6c 20 6e 61 6d 65  29 20 61 73 20 70 61 73   ail name ) as pas
    0000005B  73 77 6f 72 64 2e 0d 0a                            sword...
00000010  50 41 53 53 20 49 45 55  73 65 72 40 0d 0a         PASS IEU ser@..
    00000063  32 33 30 2d 57 65 6c 63  6f 6d 65 20 74 6f 20 74   230-Welc ome to t
    00000073  68 65 20 44 65 6c 6c 20  46 54 50 20 73 69 74 65   he Dell  FTP site
    00000083  2e 20 41 20 73 65 72 76  69 63 65 20 6f 66 20 44   . A serv ice of D
    00000093  65 6c 6c 20 49 6e 63 2e  2c 20 52 6f 75 6e 64 20   ell Inc. , Round
    000000A3  52 6f 63 6b 2c 20 54 65  78 61 73 2e 0d 0a 20 20   Rock, Te xas...
    000000B3  20 20 46 6f 72 20 69 6e  66 6f 72 6d 61 74 69 6f     For in formatio
    000000C3  6e 20 61 62 6f 75 74 20  44 45 4c 4c 2c 20 63 61   n about  DELL, ca
    000000D3  6c 6c 20 2b 31 20 38 30  30 20 39 39 39 20 33 33   ll +1 80 0 999 33
    000000E3  35 35 20 41 6c 6c 20 74  72 61 6e 73 66 65 72 73   55 All t ransfers
    000000F3  20 61 72 65 20 6c 6f 67  67 65 64 20 77 69 74 68    are log ged with
    00000103  0d 0a 20 20 20 20 79 6f  75 72 20 68 6f 73 74 20   ..    yo ur host
    00000113  6e 61 6d 65 20 61 6e 64  20 65 6d 61 69 6c 20 61   name and  email a
    00000123  64 64 72 65 73 73 2e 20  49 66 20 79 6f 75 20 64   ddress.  If you d
    00000133  6f 6e 27 74 20 6c 69 6b  65 20 74 68 69 73 20 70   on't lik e this p
    00000143  6f 6c 69 63 79 20 70 6c  65 61 73 65 20 64 69 73   olicy pl ease dis
    00000153  63 6f 6e 6e 65 63 74 20  6e 6f 77 2e 0d 0a 20 20   connect  now...
    00000163  20 20 50 6c 65 61 73 65  20 62 65 20 61 64 76 69     Please  be advi
    00000173  73 65 64 20 74 68 61 74  20 75 73 65 20 63 6f 6e   sed that  use con
    00000183  73 74 69 74 75 74 65 73  20 63 6f 6e 73 65 6e 74   stitutes  consent
    00000193  20 74 6f 20 6d 6f 6e 69  74 6f 72 69 6e 67 20 28    to moni toring (
    000001A3  45 6c 65 63 20 43 6f 6d  6d 20 50 72 69 76 20 41   Elec Com m Priv A
    000001B3  63 74 2c 0d 0a 20 20 20  20 31 38 20 55 53 43 20   ct,..     18 USC
    000001C3  32 37 30 31 2d 32 37 31  31 29 2e 20 50 6c 65 61   2701-271 1). Plea
    000001D3  73 65 20 73 65 65 20 74  68 65 20 66 69 6c 65 20   se see t he file
    000001E3  72 65 61 64 6d 65 2e 74  78 74 20 66 6f 72 20 64   readme.t xt for d
    000001F3  69 73 63 6c 61 69 6d 65  72 73 20 70 65 72 74 61   isclaime rs perta
    00000203  69 6e 69 6e 67 20 74 6f  20 74 68 69 73 0d 0a 20   ining to  this..
    00000213  20 20 20 73 65 72 76 69  63 65 2e 20 49 66 20 79      servi ce. If y
    00000223  6f 75 72 20 46 54 50 20  63 6c 69 65 6e 74 20 63   our FTP  client c
    00000233  72 61 73 68 65 73 20 6f  72 20 68 61 6e 67 73 20   rashes o r hangs
    00000243  73 68 6f 72 74 6c 79 20  61 66 74 65 72 20 6c 6f   shortly  after lo
    00000253  67 69 6e 2c 20 74 72 79  20 75 73 69 6e 67 20 61   gin, try  using a
    00000263  20 64 61 73 68 0d 0a 20  20 20 20 28 2d 29 20 61    dash..     (-) a
    00000273  73 20 74 68 65 20 66 69  72 73 74 20 63 68 61 72   s the fi rst char
    00000283  61 63 74 65 72 20 6f 66  20 79 6f 75 72 20 70 61   acter of  your pa
    00000293  73 73 77 6f 72 64 2e 20  54 68 69 73 20 77 69 6c   ssword.  This wil
    000002A3  6c 20 74 75 72 6e 20 6f  66 66 20 74 68 65 20 69   l turn o ff the i
    000002B3  6e 66 6f 72 6d 61 74 69  6f 6e 61 6c 0d 0a 20 20   nformati onal..
    000002C3  20 20 6d 65 73 73 61 67  65 73 20 77 68 69 63 68     messag es which
    000002D3  20 6d 61 79 20 62 65 20  63 6f 6e 66 75 73 69 6e    may be  confusin
    000002E3  67 20 79 6f 75 72 20 66  74 70 20 63 6c 69 65 6e   g your f tp clien
    000002F3  74 2e 0d 0a 20 20 20 20  2a 2a 2a 2a 2a 2a 2a 2a   t...     ********
    00000303  49 4e 20 43 41 53 45 20  4f 46 20 50 52 4f 42 4c   IN CASE  OF PROBL
    00000313  45 4d 53 2a 2a 2a 2a 2a  2a 2a 2a 2a 2a 2a 2a 2a   EMS***** ********
    00000323  2a 2a 2a 2a 2a 2a 2a 2a  2a 2a 2a 2a 0d 0a 20 20   ******** ****..
    00000333  20 20 2a 2a 20 46 69 6c  65 20 43 6f 6e 74 65 6e     ** Fil e Conten
    00000343  74 3a 20 73 65 6e 64 20  45 4d 41 49 4c 20 74 6f   t: send  EMAIL to
    00000353  20 64 65 6c 6c 62 62 73  40 64 65 6c 6c 2e 63 6f    dellbbs @dell.co
    00000363  6d 20 20 20 2a 2a 0d 0a  20 20 20 20 2a 2a 20 46   m   **..     ** F
    00000373  54 50 20 53 65 72 76 65  72 3a 20 73 65 6e 64 20   TP Serve r: send
    00000383  45 4d 41 49 4c 20 74 6f  20 68 6f 73 74 6d 61 73   EMAIL to  hostmas
    00000393  74 65 72 40 64 65 6c 6c  2e 63 6f 6d 20 20 2a 2a   ter@dell .com  **
    000003A3  0d 0a 20 20 20 20 2a 2a  20 57 57 57 20 53 65 72   ..    **  WWW Ser
    000003B3  76 65 72 3a 20 73 65 6e  64 20 45 4d 41 49 4c 20   ver: sen d EMAIL
    000003C3  74 6f 20 77 65 62 6d 61  73 74 65 72 40 64 65 6c   to webma ster@del
    000003D3  6c 2e 63 6f 6d 20 20 20  2a 2a 0d 0a 20 20 20 20   l.com    **..
    000003E3  2a 2a 2a 2a 2a 2a 2a 2a  2a 2a 2a 2a 2a 2a 2a 2a   ******** ********
    000003F3  2a 2a 2a 2a 2a 2a 2a 2a  2a 2a 2a 2a 2a 2a 2a 2a   ******** ********
    00000403  2a 2a 2a 2a 2a 2a 2a 2a  2a 2a 2a 2a 2a 2a 2a 2a   ******** ********
    00000413  2a 2a 2a 2a 0d 0a                                  ****..
    00000419  32 33 30 20 55 73 65 72  20 6c 6f 67 67 65 64 20   230 User  logged
    00000429  69 6e 2e 0d 0a                                     in...
0000001E  54 59 50 45 20 49 0d 0a                            TYPE I..
    0000042E  32 30 30 20 54 79 70 65  20 73 65 74 20 74 6f 20   200 Type  set to
    0000043E  49 2e 0d 0a                                        I...
00000026  50 41 53 56 0d 0a                                  PASV..
    00000442  32 32 37 20 45 6e 74 65  72 69 6e 67 20 50 61 73   227 Ente ring Pas
    00000452  73 69 76 65 20 4d 6f 64  65 20 28 31 34 33 2c 31   sive Mod e (143,1
    00000462  36 36 2c 31 31 2c 31 30  2c 32 35 31 2c 37 38 29   66,11,10 ,251,78)
    00000472  0d 0a                                              ..
0000002C  53 49 5a 45 20 2f 76 69  64 65 6f 2f 52 37 39 37   SIZE /vi deo/R797
0000003C  33 33 2e 45 58 45 0d 0a                            33.EXE..
    00000474  32 31 33 20 34 32 35 35  30 35 36 0d 0a            213 4255 056..
00000044  52 45 54 52 20 2f 76 69  64 65 6f 2f 52 37 39 37   RETR /vi deo/R797
00000054  33 33 2e 45 58 45 0d 0a                            33.EXE..
    00000481  31 32 35 20 44 61 74 61  20 63 6f 6e 6e 65 63 74   125 Data  connect
    00000491  69 6f 6e 20 61 6c 72 65  61 64 79 20 6f 70 65 6e   ion alre ady open
    000004A1  3b 20 54 72 61 6e 73 66  65 72 20 73 74 61 72 74   ; Transf er start
    000004B1  69 6e 67 2e 0d 0a                                  ing...
    000004B7  32 32 36 20 54 72 61 6e  73 66 65 72 20 63 6f 6d   226 Tran sfer com
    000004C7  70 6c 65 74 65 2e 0d 0a                            plete...

Would it not be nice if you could recreate the binary data that was transferred with FTP? Well, actually you can! And it is dead easy with tawk follow_stream() function! Let’s do it!

First, we need to find the flow where the data R79733.EXE was transferred… Remember about FTP passive mode and the ftpCDFindex column?

tawk 'flow(35) { print $ftpCDFindex }' ~/results/faf-exercise_flows.txt

36
36
tawk 'flow(36) && bitsanyset($flowStat, 0x1) { print $numBytesSnt, $tcpFStat, $tcpAnomaly }' ~/results/faf-exercise_flows.txt

numBytesSnt	tcpFStat	tcpAnomaly
4268858	        0x0c51	        0x02c4

Ok, so the data was transferred in flow 36, direction B!

But wait… there seems to be a problem… The number of bytes for flow 36 is 4268858, while the FTP SIZE command reported a size of 4255056 bytes… Let’s look at the tcpFStat and tcpAnomaly columns for this flow to understand the issue:

tawk -V tcpFStat=0x0c51 -V tcpAnomaly=0x02c4

The tcpFStat column with value 0x0c51 is to be interpreted as follows:

   bit | tcpFStat | Description
   =============================================================================
     0 | 0x0001   | Packet good for inter-distance assessment
     4 | 0x0010   | Window state-machine initialized
     6 | 0x0040   | Win 0 probe
    10 | 0x0400   | Window full
    11 | 0x0800   | Window state-machine count up(1)/down(0)


The tcpAnomaly column with value 0x02c4 is to be interpreted as follows:

   bit | tcpAnomaly | Description
   =============================================================================
     2 | 0x0004     | SEQ Fast retransmission
     6 | 0x0040     | Sequence number out-of-order
     7 | 0x0080     | Sequence mess, rather spurious Retransmission
     9 | 0x0200     | Previous packet not captured

So, it looks like we have duplicate ACK and retransmission packets and some mess with the sequence numbers. What can we do about it? We could extract the flow and sort the packets by sequence numbers… Sounds complicated… Luckily, there is actually nothing to worry about, as the follow_stream() function automatically takes care of all the nitty-gritty details! Time to extract the data!

tawk 'follow_stream(36, 3, "B")' ~/results/faf-exercise_packets.txt | xxd -p -r > ~/results/R79733.EXE

du -b ~/results/R79733.EXE

4255056 /home/user/results/R79733.EXE
file -b ~/results/R79733.EXE

PE32 executable (GUI) Intel 80386, for MS Windows, 6 sections
md5sum ~/results/R79733.EXE

6448b03e6a8709be41e7165979a440da /home/user/results/R79733.EXE

objdump -d ~/results/R79733.EXE

R79733.EXE:     file format pei-i386


Disassembly of section .text:

00401000 <.text>:
  401000:       55                      push   %ebp
  401001:       8b ec                   mov    %esp,%ebp
  401003:       81 ec 58 02 00 00       sub    $0x258,%esp
  401009:       53                      push   %ebx
  40100a:       56                      push   %esi
  40100b:       57                      push   %edi
  40100c:       68 02 7f 00 00          push   $0x7f02
  401011:       33 ff                   xor    %edi,%edi
  401013:       57                      push   %edi
  401014:       ff 15 a8 25 43 00       call   *0x4325a8
  40101a:       50                      push   %eax
  40101b:       ff 15 e4 25 43 00       call   *0x4325e4
  401021:       57                      push   %edi
  401022:       8b 75 08                mov    0x8(%ebp),%esi
  401025:       57                      push   %edi
  401026:       89 45 fc                mov    %eax,-0x4(%ebp)
  401029:       68 84 01 00 00          push   $0x184
  40102e:       56                      push   %esi
  40102f:       ff 15 e0 25 43 00       call   *0x4325e0
...

As expected, the size of the file matches that reported in the FTP control channel! A Dell graphics control firmware. Is it, or is it Marware? If you enter the MD5 into VirusTotal, you will see.

Conclusion

That is just a very brief demo of what you can do with T2 packet mode! And yes, ftpDecode is data carving capable if FTP_SAVE is enabled (refer to the FTP tutorial for more details!). So extracting R79733.EXE would be much faster in that mode. But for non data carving capable plugins, or if you want to be compatible with Wireshark output, tawk follow_stream function is definitely an option.

Don’t forget to reset t2 configuration for the next tutorials:

t2conf --reset -a && t2build -R

Have fun!