An Ultimate Guide on Network Discovery for Every Network Security Lover | Lucideus

Nowadays a big challenge in the IT sector  is to track  all the IT assets in their network because  most of the organization working on automatic delivery, for example, bill pay, employee records, live activities, punch-in - punch-out, asset record  etc so gradually the IT assets are  growing  in other terms we highly depend on IT assets . As we know if we are gaining something from technology at the same time we are facing to risk associated with it. To overcome the risk we have to maintain a parallel mechanism to avoid the risk.

Now we come on to the point where we are discussing is; Network Discovery. First of all, we want to know why it is required?

Network Discovery Requirement  

As we discussed above that in IT sector , IT assets are growing day by day so most of the organisations are facing new challenges  that how to track all assets and keep records of their assets and also track the malicious devices or rogue devices in their organisation  therefore we need a device tracking automated application that can keep all records and have the ability to find the malicious and rogue devices . we called this type of application as Network Discovery.

Discovery Methods 

  1. Keep records manually 
  2. Using built-in application NMAP agent less
  3. SNMP,SNMPTRAP agent-based 
  4. Port-mirroring using SPAN port
  5. DHCP packet capture
  6. NETFLOW ,802.1X ..etc

1. Keep records manually 
This is useful for those organization where the no of assets is very small i.e count on fingers,  so it is better to keep all the records manually instead of developing a discovery module.I n this case no use of automated network discovery module.

2. Using built-in application NMAP 
Most of the developer choose NMAP for searching IP, MAC, Hostname and the port status of devices  in the network  it is useful for debugging purpose i.e if someone wants to know some device is running or not they can use this application  but map is not much use in the organization where most of the servers are very critical because NAMP has disadvantage as given below 
  • Even if some client is live on the network there is a chance to not discover it, due to some reason like  Time expire, firewall block, ICMP block etc.
  • It consumes lots of bandwidth of the network
  • Periodic scanning require for network activity  which is not an efficient way to discover, it consumes bandwidth and CPU utilization
  • Due to periodic scanning, we can't track the exact login time of clients.
  • Scanning weaker devices and congested networks can sometimes cause an unintentional DOS or network slowdown. This can be remedied by slowing down the scan speed, which nmap allows, using the “-T” flag, and by scanning fewer machines at once.
  • Port scans are loud. There are ways to make port scans more stealthy included with Nmap, but they always require generating a whole lot of network traffic, and there is an inverse relationship between stealth and speed.
General command for scanning network device in the network

For ping sweep i.e just want to know the host reachability status in the network,  use the following command. 
#nmap -sn -n -T5 <ip-address>

 For deep scanning , information will gather like OS,Device type,NIC vendor, IP,MAC,Port Open/Closed,Hostname  following command. 
#nmap  -sS -O -F -T5 <ip-address>

If some information are not gathered using above command you can try aggressive scanning that will give more reliable information but it will take some execution delay. 
#nmap  -A  <ip-address>

Some time using nmap you will not get the hostname of the device, some exceptional case, for example, if the device is wifi access point or web server, in that case, you may use wget command for getting information from these devices.  we will discuss namp discovery method in Passive Scanning using NMAP further in this article.

3. SNMP, SNMPTRAP agent-based

This method is more reliable than NMAP, nowadays all most all network devices having SNMP feature that can send their information to the central SNMP manager the information could be IP address, MAC, Interface UP/Down, CPU utilization, Memory uses, etc. 

The Process of SNMP TRAP setting 
  • First of all, go to the SNMP enabled device and add the manager IP address for receiving their information.
     
  • Install net-SNMP on manager device and configure it to capture the notification of devices connected to a network.
  • By default, the SNMP trap notification will be saved in Syslog so change the log path to your log.
  • The command for generating trap  version 1 
#sudo snmptrap -v 1 -c public 127.0.0.1 .1.3.6.1 localhost 6 17 '' .1.3.6.1 s "Just a test"                
when you will execute this command a trap will generate on your manager system just ensure your system is capable to capture the SNMP trap request. 

  •  Command for initiating the SNMP trap daemon on manager system
            #sudo snmptrapd  -f -Lf  trap_data.txt

where trap_data.txt is the file where you can capture the traps, before executing this command ensure that the default SNMP trap call is not running  using the command     
 # /etc/init.d/snmptrapd status if it is running then kill them first.
  
  • After SNMP trap setting on manager and client side, you are able to capture the traps for the related information.T he traps information are based on OID so you have to parse table for OID to corresponding information. The OID  information you can parse using SNMPWALK command. Let us assume our client device is a switch and we want information about the connected host to switch. The following sequence of command will execute on manager side.  
#snmpwalk -mALL -c public   -v1  <ip-adds>            ifDescr
#snmpwalk -mALL -c public   -v1  <ip-adds>            ifOperStatus
#snmpwalk -mALL -c public   -v1  <ip-adds>            dot1dTpFdbPort
#snmpwalk -mALL -c public   -v1  <ip-adds>            dot1dTpFdbAddress

based on above commands output we can prepare a record for connected device on the switch as shown below.
  • Now we have a list of connected device’s MAC address from above table  and  also we have ARP table where IP-MAC records are maintained, by evaluating these two tables we can discover the connected and disconnected devices from the network.
ID
IntfID
IntfName
Status
MAC_bind
Unq_STRING
748
49
gigabitethernet1
UP
f4:8c:50:fb:2a:bb
..P.*.
749
50
gigabitethernet2
DOWN
f4:8c:50:76:e8:eb
..T.)

4. Port-mirroring using SPAN port
Port mirroring is a robust mechanism for discovering the network devices , rouge devices in the network .Using Port-mirroring we can capture the entire network data for example, it will capture  IP packets and using packet parsing mechanism we can find the IP-MAC  from  the header  and based on this IP we can find the host name on the authentication server (Radius server ) and using  some signatures(TTL time ,packet size ,DF flags ..) we can find the OS types,  hence we can find all informations regarding the captured  IP. 

5. Passive scanning using DHCP server

As we know whenever an endpoint come into the network and try to connect,  it first requests to DHCP for providing them with an IP address.

#Source IP 0.0.0.0 and destination IP 255.255.255.255  with source MAC xx:xx:xx:xx:xx:xx
from source port 67 to destination port 68 . It is a UDP broadcast traffic in the network.based on above information we can gather many pieces of information.

Following list of the process for.

1. We have a packet sniffing application that can continue watching and catch the broadcast packet from source port 67 and packet type UDP.

2. If we have DHCP credentials then access the file name.
var/lib/dhcpd/dhcpd.leases and parse it .It has information, mentioned in the problem statement, about all connected endpoints and stores this information in BUFFER in running application.

3. Lets us assume our network subnet size 255 i.e we have 255 endpoints in our network.

4. DHCP allocate every endpoint an IP for some Days or Months .i.e for the same MAC address, IP will be same.

5. Now our sniffing function starts watching and once they capture a DHCP request it extracts source MAC address from the request.N ow two thing can happen

  • The received MAC address already stored in BUFFER: In this case, we have to just find all information from BUFFER for the matched MAC address.
  • The received MAC address does not find in the BUFFER: Then first it connects to the DHCP server and bring all newly connected device information and update the BUFFER.O nce the buffer has been updated the received MAC will match with BUFFER and display the information.
We have taken two approach for packet sniffing Using raw socket : in this case we have a raw socket and on that socket whenever a UDP packet come the parent process create a child process and assign further work to them. Using PCAP function.

The is a  packet  format for dhcp request.
0x0030:  35d1 0000 0000 0000 0000 0000 0000 0000  5...............
0x0040:  0000 0000 0000 748d 082b 4fff 0000 0000  ......t..+O.....
0x0050:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x0060:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x0070:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x0080:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x0090:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x00a0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x00b0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x00c0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x00d0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x00e0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x00f0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x0100:  0000 0000 0000 0000 0000 0000 0000 0000  ................
0x0110:  0000 0000 0000 6382 5363 3501 0337 0701  ......c.Sc5..7..
0x0120:  7903 060f 77fc 3902 05dc 3d07 0174 8d08  y...w.9...=..t..
0x0130:  2b4f ff32 04c0 a800 4c33 0400 76a7 000c  +O.2....L3..v…
0x0140:  0e44 6576 6573 6873 2d69 5068 6f6e 65ff  .Deveshs-iPhone.
0x0150:  0000 0000 0000

Every DHCP packet is divided into three categories 
  • Code   
  • Length 
  • Message
At 28th location of DHCP packet we can find the MAC address of client, in  above packet format we can see the MAC address  748d 082b 4fff . Some basic code meaning in DHCP Packet

CODE 50, means the message part hold the IP address.
CODE 12 means the message part  hold the HOST name.
CODE 54 means the DHCP server IP Provider.
CODE 61 means the message part will me MAC address for mobile device.
CODE 53 means this is DHCP packet
And CODE 3 means it is requested packet.
Capturing DHCP packet Algorithm data is an array variable that hold the dhcp packet data .

if (  data[ 240 ] == 53 && data[ 242 ] == 3 )  {
            /* Looping for capture code and options */
            for( i = 240  ;  i < Size ; )
            {
                    code = data[ i ];
                    len =  data[ i + 1 ];
                                 /* CODE 61 means the message part will me MAC */
                    if ( code  ==  61 ) { address for  mobile  device
                            mobile = 1;
                    }
                    /* CODE 50 means the message part hold the IP address */
                               else if ( code == 50 ) { 
                            sprintf( ip,"%d.%d.%d.%d", data[i+2],data[i+3],data[i+4],data[i+5] );
                    }
                                /* CODE 54 means the DHCP server IP Provider */
                    else if ( code == 54 ) { 
                            sprintf(ip_dhcp,"%d.%d.%d.%d",data[i+2],data[i+3],data[i+4],data[i+5]);
                    }
                              /* CODE 12 means the message part  hold the HOST name */
                    else if ( code == 12 ) { 
                                len = data[i+1];
                            strncpy(host_name,&data[i+2],len );
                    }
                    i = i + len + 2;
}

Below diagram is for live detection of any client in the network


6. Passive Scanning using NMAP
a) Firstly we program an efficient code for executing nmap’s deep scan within 60 Second .

b) Execute that efficient program and gather all information about the current connected end-points in the network. For example nmap -Pn -O -F -T5 <ip-address>.

c) Store all information about connected endpoints in the BUFFER “nmap_rec_dscan[255]”.
structure details

struct nmap_deep_scan_node {                                                                                                              time_t  av_time;
time_t  login_time;                       // Device Login time in network
time_t  logout_time;                   // Device Logout time in network
char ip[SIZE];
char mac[SIZE];
char os_type[EXTRA_SIZE];
char dev_type[AVG_SIZE];        // generic,switch...
char dev_catg[AVG_SIZE];       // switch,router,printer,laptop,wifi dev..
char dev_name[GEN_SIZE];    // cisco,fortinet
char host_name[SIZE];             // Laptop Hostname,  Hari,Manish,Lenovo when DHCP capture it store into host_name
char port_name[AVG_SIZE];
char ip_dhcp[SIZE];
int  status;                                     // Means it has been traced by discovery module previously ; 1 learned 0 not learned 
int  score;
int  hid;
int  live_run;                                 // It means PCAP is running and packet capture at live time; 0 not run 1 run 
int  nmap_run;                            // It means nmap running initial;  0 not run 1 run  
int  con;                                         // con =1 means UP and =0 means DOWN 

};

Execute nmap ping sweep using the following command.It will take maximum 1.5 Second nmap -sn -n -T5 <ip-address>.
Parse the information and store it into BUFFER  “nmap_rec_pswip[255]”.
Structure details.
struct nmap_pswip_node {
char ip[SIZE];
char mac[SIZE];
int status;
};

Now call the sniffing function (pcap module) and watch , if any host approaching to DHCP request , it will capture it’s MAC address and search (sequential ) into namp_rec_dscan[255] BUFFER and if it will find in the BUFFER then the Host is old one which is logging again and if it does not find in BUFFER it will be a new Host connecting to network.

If MAC is not found in the BUFFER namp_rec_dscan[255] then the process call to the ping_swip function and gather information about new host within 1.5 seconds and it will update the namp_rec_pswip[255] BUFFER i.e IP and MAC, One IP will know to the MAC it will execute a deep_scan function in 2-3 second and update information about the new host in BUFFER namp_rec_dscan[255].

7.0 Fast deep scan using NMAP API
Using  /Thread /Scheduled Thread/Scheduled Process :
Details: If we execute the following command for network deep scan using NMAP it will take around 5-6 Minute.

nmap -Pn -O -F -T4 192.168.0.0/24 i.e 255 hosts will be scanned in a 5-6 minute that is very time-consuming.

we will programme a fast code to scan the 255 host within 70 Second.
Declare a global buffer NMAP_REC_DSCAN[255].
struct nmap_deep_scan_node 

{                                                                                                                                                             time_t  av_time;
time_t  login_time;               // Device Login time in network
time_t  logout_time;            // Device Logout time in network
char ip[SIZE];
char mac[SIZE];
char os_type[EXTRA_SIZE];
char dev_type[AVG_SIZE];    // generic,switch...
char dev_catg[AVG_SIZE];     // switch,router,printer,laptop,wifi dev..
char dev_name[GEN_SIZE];   // cisco,fortinet
char host_name[SIZE];            // Laptop Hostname,  Hari,Manish,Lenovo when DHCP capture it store into host_name
char port_name[AVG_SIZE];
char ip_dhcp[SIZE];
int  status;                                     // Means it has been traced by discovery module previously ; 1 learned 0 not learned 
int  score;
int  hid;
int  live_run;                                 // It means PCAP is running and packet capture at live time; 0 not run 1 run 
int  nmap_run;                            // It means nmap running initial;  0 not run 1 run 
int  con;                                         // con =1 means UP and =0 means DOWN 
};

Now create 20 thread and assign every thread a unique number that unique no is between 0-255. 
  • On every thread has a dedicated IP based on the unique no, for example, thread 1 has unique no 0 so it has ip assigned 192.168.0.1, last octet (0+1).
  • Once every thread complete the nmap command the other function parse it and store the host related information in BUFFER NMAP_REC_DSCAN[255].
  • Overall it will take 70 Second to scan the entire network.
  • Scheduled the thread into 50 threads, at one time only 50 thread can execute NMAP command. it can be set according to the server’s RAM and Processor
8. Packet sniffing approach

Details: I have taken two approaches for packet sniffing

  • Using the Raw socket(process and thread ).
  • Using Pcap (Process and thread ).
9. Using Raw Socket and child Process
  • Created Raw socket for listening packets coming on  interface                                                        sock_raw = socket( AF_PACKET , SOCK_RAW , htons(ETH_P_ALL)) .
  • Once a packet arrive on interface i.e the parent process receive packet using recvfrom system call
  • data_size = recvfrom(sock_raw , buffer , 65536 , 0 , &saddr , (socklen_t*)&saddr_size);

10. Using the Raw socket and multiple Thread
  • Create the Raw socket for listening packets coming on the interface.
  • Once a packet arrives on the interface  the parent process receive packet using recvfrom system call and multiple child threads who have assigned an  IP address will  send the nmap result to the parent 
Code snippet for creating threads

if(data_size > 0 ) {
//printf("calling new thread\n");
args.arg1 = buffer;
args.arg2 = data_size;
args.arg3 = packetTyp;

int err = pthread_create( &tid, NULL, & dosomething, (void *)&args );
if(err != 0) {
printf("\n can't create thread -error\n");
}

11. Using Pcap Process 
Create a pcap sniffing program   
Code snippet  for pcap sniffing.

handle = pcap_open_live( interface , 65536 , 0 , 0 , errbuf );
    if ( handle == NULL ) {
            fprintf( stderr, "Couldn't open device %s : %s\n" , interface , errbuf );
            exit(1);
    }
/** filter packet for specific port using pcap_compile function **/
if ( pcap_compile( handle , &fp , filter_exp , 0 , net ) == -1 ) {
            perror("pcap_compile:error");
            return;
    }
    if ( pcap_setfilter( handle, &fp ) == -1 ) {
            perror("pcap_setfilter:error");
            return;
    }
    /* Put the device in sniff loop */
    pcap_loop( handle , -1 , process_packet , ptr );
Once a packet arrive at interface, pcap capture it and parse the packet .
Code snippet for capture packet 
void process_packet( u_char *args, const struct pcap_pkthdr *header, const u_char *buffer )
{
    int size = header->len;
    char sip[50]; //source IP
    char tip[50]; // target IP
    char smac[50];   // Source MAC
    char tmac[50];   // target MAC

    arphdr_t *arpheader = NULL;
    unsigned short iphdrlen;
    time_t t = time( NULL );
    memset(sip,0,sizeof(sip));
                memset(tip,0,sizeof(tip));
    memset(smac,0,sizeof(smac));
    memset(tmac,0,sizeof(tmac));
    const struct pcap_pkthdr *packet  = header;
    struct iphdr *iph = ( struct iphdr * )( buffer +  sizeof( struct ethhdr ) );
    /* PACKET length  = ether_header(14 Byte) + IP_header(20 Byte) + UDP_header(8 Byte)  + data*/  
                /*  capture UDP packet */
  if( iph->protocol == 17 ) {  
iphdrlen = iph -> ihl * 4;
struct udphdr *udph = ( struct udphdr* )( buffer + iphdrlen  + sizeof(             struct ethhdr ) );
int header_size =  sizeof( struct ethhdr ) + iphdrlen + sizeof udph;
     
/* call function for checking the captured packet is DHCP request or not*/
PrintData( buffer + header_size , size - header_size , args );
                      }
    else {
  int i=0;
            arpheader = ( struct arphdr * )( iph );
                        /* If is Ethernet and IPv4, print packet contents */
            if ( ntohs( arpheader -> htype ) == 1 && ntohs( arpheader -> ptype ) == 0x0800 ){
 sprintf( tip , "%d.%d.%d.%d" , arpheader->tpa[0] , arpheader->tpa[1] , arpheader->tpa[2] , arpheader->tpa[3] );
sprintf( sip , "%d.%d.%d.%d" , arpheader->spa[0] , arpheader->spa[1] , arpheader->spa[2] , arpheader->spa[3] );
sprintf( tmac , "%02X:%02X:%02X:%02X:%02X:%02X", arpheader->tha[0] , arpheader->tha[1] , arpheader->tha[2] , arpheader->tha[3] , arpheader->tha[4] , arpheader->tha[5] );
sprintf( smac , "%02X:%02X:%02X:%02X:%02X:%02X", arpheader->sha[0] , arpheader->sha[1] , arpheader->sha[2] , arpheader->sha[3] , arpheader->sha[4] , arpheader->sha[5] );
            }
if( strcmp ( sip , "192.168.0.1" ) == 0 && strcmp ( tmac , "00:00:00:00:00:00" ) == 0  )  {
send_tip_to_parent_process( tip , t ,  args )
}
}
}

References
https://www.slideshare.net/Netmanias/20131004dhcp-message-format
https://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml
https://tools.ietf.org/html/rfc2132
http://net-snmp.sourceforge.net/wiki/index.php/TUT:Configuring_snmptrapd_to_receive_SNMPv3_notifications
https://support.nagios.com/kb/article.php?id=88
https://www.forescout.com/wp-content/uploads/2015/12/ForeScout-CounterACT-CDM-Whitepaper.pdf

No comments:

Powered by Blogger.