Showing posts with label Troubleshooting. Show all posts
Showing posts with label Troubleshooting. Show all posts

Tuesday, 22 July 2025

Troubleshooting EIGRP !!

1️⃣ Mismatched Autonomous System (AS) Numbers

Description: Both routers must be in the same EIGRP AS.

Fix:

router eigrp 100

2️⃣ Interface in the Wrong VRF

Description: If the interface is in a VRF and EIGRP isn’t running in that VRF, the neighbor won’t form.

Fix: Configure EIGRP under the correct VRF context.


3️⃣ Passive Interface Enabled on EIGRP-Connected Interface

Description: Passive interface disables EIGRP hello packets.

Fix:

router eigrp 100 no passive-interface e1/0

4️⃣ MTU Mismatch

Description: EIGRP includes MTU in its hello packets. If received MTU is less than interface MTU, neighborship fails.

Fix: Ensure MTU matches on both sides of the WAN link.


5️⃣ K-Values Mismatch

Description: Routers with different metric calculation settings (K-values) will not form neighbors.

Fix:


router eigrp 100 metric weights 0 1 0 1 0 0 ! default

Or:


default metric weights

6️⃣ Authentication Mismatch

Description: MD5 authentication must be enabled with same key-string on both routers.

Fix:


key chain AUTHKEY key 1 key-string Cisco123 ! interface e1/0 ip authentication mode eigrp 100 md5 ip authentication key-chain eigrp 100 AUTHKEY

7️⃣ Interface Not in the network Command

Description: If the interface is not covered by the network command, EIGRP won’t operate on it.

Fix:


router eigrp 100 network 12.1.1.0 0.0.0.3

8️⃣ Interface is Down or Administratively Shut

Description: Basic, but often overlooked — if the interface is down, EIGRP won’t send or receive hello packets.

Fix:


interface e1/0 no shutdown

9️⃣ IP Address Misconfiguration

Description: Wrong IP address — routers are not in the same subnet.

Fix: Assign valid IPs in the same subnet on both sides.


๐Ÿ”Ÿ Access Control List (ACL) Blocking EIGRP

Description: ACLs on the interface or VTY lines may block EIGRP traffic (uses IP protocol number 88).

Fix: Allow EIGRP in ACL:


permit eigrp any any

1️⃣1️⃣ Duplicate Router IDs

Description: Duplicate router IDs won’t stop neighbor formation, but can cause route instability or flapping.

Fix:


router eigrp 100 eigrp router-id 1.1.1.1

Ensure unique router IDs.


1️⃣2️⃣ Hello/Hold Timer Mismatch (older IOS versions)

Description: If timers differ significantly, routers may not recognize each other as valid neighbors.

Fix:


interface e1/0 ip hello-interval eigrp 100 5 ip hold-time eigrp 100 15

Set consistently on both routers.


1️⃣3️⃣ Unidirectional Link (Physical/Cabling Issue)

Description: One router sends hellos, but the other can’t receive — often due to cable, NIC, or virtual lab misconfig.

Fix: Check cabling or rebuild link in virtual environments.


1️⃣4️⃣ EIGRP Not Enabled for IPv6

Description: For IPv6, you must use ipv6 router eigrp, and EIGRP is configured per interface, not via network.

Fix:


interface e1/0 ipv6 eigrp 100


๐Ÿงช Verification Commands

CommandWhat It Shows
show ip eigrp neighborsNeighbor status
show ip protocolsAS, timers, passive interfaces
debug eigrp packetsHellos, updates, and errors
debug eigrp neighborsAdjacency process
show running-configAuthentication, K-values, interfaces

Network Engineer Stuff

Saturday, 31 May 2025

Troubleshooting Packet Drops in a Switch

When packets are getting dropped on a switch, it can result from various issues across different layers. Here's a structured way to troubleshoot packet drops on a switch:


๐Ÿ” Troubleshooting Packet Drops in a Switch


1. Check Interface-Level Counters

Use the following commands (based on switch OS):

  • Cisco IOS:

    show interfaces [interface-id]
    show interfaces counters errors
    show interfaces status err-disabled
    
  • Look for:

    • Input/output errors

    • CRC errors (Layer 1 issue)

    • Collisions (Half-duplex problems)

    • Drops (buffers exhausted)


2. Verify Port Configuration

  • Speed/Duplex mismatch:

    show interfaces [interface-id] status
    show running-config interface [interface-id]
    
    • Ensure both sides of a link have the same speed/duplex settings.

    • Autonegotiation issues can cause excessive drops.


3. Check for Congestion or Buffer Overflows

  • Causes:

    • High traffic load on uplinks

    • Microbursts

    • Insufficient buffer allocation

  • Commands:

    show platform hardware capacity [interface/buffer stats]
    show queueing interface [interface-id]      (on some platforms)
    
  • Solution:

    • Use QoS to prioritize important traffic.

    • Increase buffer sizes (if configurable).

    • Load-balance traffic over multiple links (EtherChannel).


4. Look for Broadcast Storms or Loops

  • Symptoms:

    • High CPU usage

    • Drops on multiple interfaces

    • MAC flaps

  • Commands:

    show mac address-table
    show spanning-tree
    show processes cpu sorted
    
  • Fixes:

    • Enable/verify Spanning Tree Protocol (STP)

    • Enable Storm Control

    • Check for loopback cables or misconnected devices


5. Inspect QoS Policies

  • Misconfigured QoS can lead to packet drops in input/output queues.

  • Commands:

    show policy-map interface [interface-id]
    
  • Check for:

    • Drop counters under QoS class-maps

    • Policing or shaping issues


6. CPU or Control Plane Congestion

  • Some traffic gets punted to CPU (e.g., ARP, STP BPDUs). Excessive control traffic can overwhelm the switch CPU.

  • Commands:

    show processes cpu
    show platform cpu packet statistics
    
  • Fixes:

    • Apply CoPP (Control Plane Policing)

    • Offload traffic processing if possible


7. Hardware Failures or Bugs

  • Bad interfaces, line cards, or known software bugs can also cause unexplained drops.

  • Steps:

    • Check logs: show logging

    • Check for hardware errors: show environment, show module

    • Search vendor bug database (e.g., Cisco Bug Toolkit)


๐Ÿ”ง Summary of Common Causes

Issue Type  Common Symptoms Quick Fix
Duplex mismatch  CRC errors, collisions Match speed/duplex manually
Congestion   Interface drops Use QoS, upgrade link
STP/loops CPU spike, flooding Enable STP, storm control
QoS misconfig Output queue drops Tune policies, verify classes
Hardware/bugs Unexplained drops RMA or firmware upgrade


Saturday, 8 August 2020

Enabling SSH

 Troubleshoot Ticket 


Network Team uses telnet (port 23) to login in the device 10.1.1.1

Need to restrict the use of telnet since telnet is not secure ; session is not encrypted

Enable ssh (port 22) which is secure protocol and where the session is encrypted

Can be configured in working hours since no downtime is required 

========================================================

Lets open Putty and try to login via ssh to device 10.1.1.1

Here we can see we are getting error - Connection refused which means ssh is not enabled on the device

10.1.1.1 and to solve it we need to enable ssh 


Now lets try to login via telnet 


 

We are able to login via telnet !

 

So lets enable SSH 

Telnet session is insecure because the information is in the clear text 

For SSH session that is not the case , for ssh session the information is encrypted  .

To encrypt / decrypt any information or data we need to generate certain keys which are associated with the hostname and domain name . 

 So lets first configure a hostname and domain name for the device

R1(config)#hostname PUNE_ROUTER
PUNE_ROUTER(config)#
PUNE_ROUTER(config)#ip domain name amartechstuff.net
PUNE_ROUTER(config)#

Note:- I am just configuring a random domain name . If you are working in real environment your 

organization will have valid public domain name .

Now lets  move forward and generate crypto keys which will be used to encrypt / decrypt the information 

PUNE_ROUTER(config)#crypto key generate rsa
The name for the keys will be: PUNE_ROUTER.amartechstuff.net
Choose the size of the key modulus in the range of 360 to 4096 for your
  General Purpose Keys. Choosing a key modulus greater than 512 may take
  a few minutes.

How many bits in the modulus [512]: 1024
% Generating 1024 bit RSA keys, keys will be non-exportable...
[OK] (elapsed time was 1 seconds)

PUNE_ROUTER(config)#
*Aug  8 12:16:18.659: %SSH-5-ENABLED: SSH 1.99 has been enabled
PUNE_ROUTER(config)#^Z

You can see that the name of the key is PUNE_ROUTER.amartechstuff.net i.e 

hostname + domain name 

Lets see the key 

PUNE_ROUTER#sh ip ssh
SSH Enabled - version 1.99
Authentication timeout: 120 secs; Authentication retries: 3
Minimum expected Diffie Hellman key size : 1024 bits
IOS Keys in SECSH format(ssh-rsa, base64 encoded):
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDfHDOhreEAVbaifqV56LS1qTGicGEC+2s0EqRwA4Ef
qIh8DS029RxkaLYll4iKlxkdJ6Qu/TphdbefEmpSidzVu+gJ80NUUxmlpiMGtF0SY5Yzd6VCsvyRFQo8
dwsdioq686NK+UpYGh46mM3GxP2cDKuvT7ytdDlBSgnsWKn04w==
PUNE_ROUTER#


Now the next and last step is to check and enable ssh under line vty 

line vty 0 4
 login local
 transport input telnet
line vty 5 15
 login local
 transport input telnet

Now we can see that telnet is configured .

Lets change it to ssh

PUNE_ROUTER#config
Configuring from terminal, memory, or network [terminal]?
Enter configuration commands, one per line.  End with CNTL/Z.
PUNE_ROUTER(config)#line vty 0 15
PUNE_ROUTER(config-line)#transport input ssh
PUNE_ROUTER(config-line)#^Z

Now lets check whether ssh is enabled or not 



 

 
https://networkengineerstuff.blogspot.com





Thursday, 12 March 2020

Troubleshooting High CPU Utilization for Routers & Switches

Lets discuss how to troubleshoot High CPU utilization for Routers and Switches


For troubleshooting the issue we must try to find which all  processes  running on the device are causing the CPU utilization .

TROUBLESHOOTING HIGH CPU UTILIZATION 


A very good commands to use is  "show cpu process sorted" which shows you how busy is the CPU for last 5 secs , 1 min and 5 mins .

The commands also shows the CPU utilization each process has consumed over these period of time .

R1#show processes cpu sorted
CPU utilization for five seconds: 1%/0%; one minute: 1%; five minutes: 1%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 103        6000      3322       1806  0.73%  0.62%  0.62%   0 encrypt proc
  38        1692       264       6409  0.16%  0.14%  0.16%   0 Compute load avg
   2           4       263         15  0.00%  0.00%  0.00%   0 Load Meter
   3          76        45       1688  0.00%  0.00%  0.00%   0 CEF Scanner
   4           0         1          0  0.00%  0.00%  0.00%   0 EDDRI_MAIN
 < output omitted>

Another useful command is "show cpu process history"

This command shows the graphical representation of the cpu utilized  in last 60 seconds , 60 minutes and 72 hours which helps us to analyse the cpu utilization time ,

R1#show processes cpu history

R1   10:00:03 AM Tuesday Jul 2 2019 UTC



    2     1111111111111111111111111     111111111111111     1111
100
 90
 80
 70
 60
 50
 40
 30
 20
 10
   0....5....1....1....2....2....3....3....4....4....5....5....6
             0    5    0    5    0    5    0    5    0    5    0
               CPU% per second (last 60 seconds)


                                    9
    23211121212222122212112114511 341
100
 90                                 *
 80                                 *
 70                                 *
 60                                 *
 50                                 *
 40                                 *
 30                                 *
 20                                 *
 10                           *     #
   0....5....1....1....2....2....3....3....4....4....5....5....6
             0    5    0    5    0    5    0    5    0    5    0
               CPU% per minute (last 60 minutes)
              * = maximum CPU%   # = average CPU%




100
 90
 80
 70
 60
 50
 40
 30
 20
 10
   0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
             0    5    0    5    0    5    0    5    0    5    0    5    0
                   CPU% per hour (last 72 hours)
                  * = maximum CPU%   # = average CPU%


There are multiple reasons why the cpu utilization would go high 

1) Deubgging is On can cause CPU utilization go high .

undebug all 

is the command to stop debugging on Router and Switches

2) ARP Input Processes -Router may originate an excessive number of ARP requests.

3) Interface/s flapping - an continuous flapping interface causing continuous changes in the Routing Table can cause excessive CPU Utilization.

4) TCP Process  -Large number of TCP session established on the Router can also be a reason for high CPU Utilization.

5) BGP Scanner - High CPU due to the BGP scanner process can be expected for short durations on a router carrying a large Internet routing table. Once a minute, BGP scanner walks the BGP RIB table and performs important maintenance tasks. These tasks include checking the next-hop referenced in the router's BGP table and verifying that the next-hop devices can be reached. Thus, a large BGP table takes an equivalently large amount of time to be walked and validated.

Because the BGP Scanner process runs through the entire BGP table, the duration of the high CPU condition varies with the number of neighbors and the number of routes learned per neighbor. Use the show ip bgp summary and show ip route summary commands to capture this information.

6) Exec & Virtual Exec Process-The Exec process is responsible for communication on the tty lines (console, auxiliary, asynchronous) of the router.

The Virtual Exec process is responsible for the vty lines (telnet sessions).

R1#show process | i CPU|Exec
CPU utilization for five seconds: 1%/0%; one minute: 1%; five minutes: 1%
  31 M*         0         2272        651    3490 9728/12000  0 Exec
R1#


The CPU utilization for the Exec process increases if there are lot of data transferred through these sessions

For the console (Exec), the router uses one interrupt per characte

The console interrupt can be seen in the show stacks command output:

R1#show stacks
Minimum process stacks:
 Free/Size   Name
 5556/6000   Clock Update Proc
 5636/6000   Inspect Init Msg
 2612/3000   allegro libretto init
 3348/12000  Init
59416/60000  script background loader
 5448/6000   RADIUS INITCONFIG
 2536/3000   Rom Random Update Process

Interrupt level stacks:
Level    Called Unused/Size  Name
  3           0   9000/9000  PA Management Int Handler
  4       57475   6912/9000  Network interfaces
  5           0   9000/9000  Timebase Reference Interrupt
  6        4355   8896/9000  16552 Con/Aux Interrupt ===>check for unused 
  7     1925809   8896/9000  MPC860T TIMER INTERRUPT


  • Disable console logging on the router with (no logging console ).
  • Verify if a long output is printed on the console (for eg., a show tech-support or a show memory ).
  • The exec command is configured for asynchronous and auxiliary lines. If a line has only outgoing traffic, the Exec process should be disabled for this line, because if the device (for example, a modem) attached to this line sends some unsolicited data, the Exec process starts on this line.If the router is used as terminal-server (for reverse telnet to other device consoles), it is recommended that you configure no exec on the lines that are connected to the console of the other devices.Data that comes back from the console might otherwise start an EXEC process, which uses CPU resources.



For the vty line (Virtual Exec), the telnet session has to build a TCP packet and send the character(s) to the telnet client.
     
       If huge amount of data is transfer through the vty ports CPU utilization may go high

The commands to verify the amount of data transferred is 

"show tcp vty 0"



Commands used to collect more information

show processes cpu Command
show interfaces Command
show interfaces switching Command
show interfaces stat Command
show ip nat translations
show align Command
show version Command
show log Command