Suddenly Proxy stop answering to Users TCP/MISS 503 error

Error : Users are experiencing web site surfing issues with the proxy. They can not surf any website and in the access.log file all the requests have logged as TCP/MISS 503 error message.

To rectify the issue try with the below workarounds.

1. Clear the squid cache folder located in /var/spool/squid
2. Rearrange the ACLs with new configuration.

But issue remains. Then I have given full access to all the users to confirm the issue. Once users get the full access they able to access the Internet. With that it's confirmed that issue is with the proxy.

While monitoring the access.log it found that users were able to access the sites with the IP address. Then able to figure out the root cause is that proxy failed to resolve Hostname to IP address.

Solution :

checked the squid configuration file. /etc/squid/squid.conf

It has three name servers configured.

I have changed the name servers to google DNS, then restart the proxy server. Finally it fixed and users able to surf Internet through proxy.

Un-install Zimbra from CentOS system

I have gone through a Zimbra installation on CentOS 6.7 Operating System and made a mistake with the installation. Therefore, it has a requirement of reconfiguration of Zimbra. As far as I have understood, if there are any configured files for Zimbra it could not perform reconfiguration.

The solution I have found was completely remove the Zimbra installation from the CentOS and install it from the beginning.

As root:

Run the commands:
su – zimbra
zmcontrol stop
exit

Then Login as Root user to the system and perform below command.

Run the command: ps -ef | grep -i zimbra
If you see running processes
Kill any lingering processes: kill -9 <pid>

Run the command: df
If you see "amavisd" with the output then you need to unmount the amavisd folder. But most cases, it does not have seperate mount point. If tha case then skip this step.
run command: umount /opt/zimbra/amavisd<-new-blah>/tmp

cd /install_dir/zcs/

Run the command: ./install.sh -u

Run the following commands to complete the unistall:
rm -rf /opt/zimbra
rm -rf /var/log/*zimbra*
rm -rf /tmp/*zimbra*
rm -rf /tmp/hsperfdata*
rm -rf /tmp/install.*
rm -rf /tmp/*swatch*
rm -rf /tmp/log*

Ensure that you removed ALL the files owned by the user zimbra and that contain the name "zimbra" in:
/var/log/
/tmp/

Even after deleting if there are any folder remain delete it.

Run the following commands to delete the users and groups:

userdel zimbra
userdel postfix
groupdel zimbra
groupdel postfix

Remove the Zimbra logging lines below from /etc/rsyslog.conf:
local0.* -/var/log/zimbra.log
auth.* -/var/log/zimbra.log
mail.* -/var/log/zimbra.log

But most probably those parameters are not there. Again if that is the case skip this step.

Remove logrotate config file for Zimbra (CentOS, RHEL)
rm -f /etc/logrotate.d/zimbra

Remove the Zimbra lines below from /etc/prelink.conf (CentOS, RHEL)
# added for Zimbra
-l /opt/zimbra/lib
-l /opt/zimbra/sleepycat/lib
-l /opt/zimbra/openldap/lib
-l /opt/zimbra/cyrus-sasl/lib
-l /opt/zimbra/mysql/lib

Remove Zimbra entries in /etc/rc* (CentOS, RHEL)
chkconfig --del zimbra

With this you are done, you can confirm this with “rpm -q zimbra”

VMware Virtual Center Server Stopped

Root Cause

While I go through the Event viewer I have found several error logs in the System and Application. System Log have Event ID 7024 which is related to VMware VCenter Server.

“The VMware VirtualCenter Server service terminated with service-specific error the system cannot find the file specified.”

Then checked the application log since above error is not obvious and cloud not imagine exact reason for the error.

By looking at the Application log it’s having lot of errors with Event ID 1105 and below is the detail of the error event

Log Name: Application

Source: MSSQL$SQLEXP_VIM

Date: 01/25/2016 14:30:48

Event ID: 1105

Task Category: Server

Level: Error

Keywords: Classic

User: SYSTEM

Description:

Could not allocate space for object ‘dbo.VPX_HOST_VM_CONFIG_OPTION’.'PK_VPX_HOST_VM_CONFIG_OPTION’ in database ‘VIM_VCDB’ because the ‘PRIMARY’ filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.

I have gone through database and the size of the database is 9.99GB. Microsoft SQL Server 2008 R2 is only supporting up to 10 GB. This is the root cause for the issue.

Solution

Running the following from the commands to shrink the database.

cd \

cd Program Files\Microsoft SQL Server\100\Tools\Binn\

OSQL.EXE -S localhost\VIM_ SQLEXP -E

Then follow the below, update value with your desired time to keep items for inside the DB.

1> use VIM_VCDB

2> go

1> update vpx_parameter set value=’30’ where name=’event.maxAge’

2> update vpx_parameter set value=’30’ where name=’task.maxAge’

3> update vpx_parameter set value=’true’ where name=’event.maxAgeEnabled’

4> update vpx_parameter set value=’true’ where name=’task.maxAgeEnabled’

5> go

(1 row affected)

(1 row affected)

(1 row affected)

(1 row affected)

1> exec cleanup_events_tasks_proc

2> go

1> dbcc shrinkdatabase (“VIM_VCDB”)

2> go

1> quit

Then start the VMware Virtual Center Server service.

SSH blocks on 'expecting SSH2_MSG_KEX_DH_GEX_GROUP'

Recently I have updated one of the server to CentOS 7. It have access to several network. With the previous CentOS version, it had SSH access to those networks. But with the server upgrade the server was not able to access via SSH.

To troubleshoot the issue I have run ssh with the the verbose mode.

#ssh -vvv <Remote server IP> -l <user name>

Then session got stuck in SSH2_MSG_KEX_DH_GEX_GROUP message.

If I briefly explain the network setup, there is a intermediate firewall which is controlling all the network traffic between those VLANS. I found by googling that it's something related to the packet fragmentation.

By default Network card MTU ( Maximum transfer unit ) is 1500. Then I have run the below command to change the MTU of the network card.

#ifconfig <network_device_name> mtu 576 ( You can try with different MTU 1000, 1472, or 567 )

Then I tried to ssh from the other network, It worked. If you want more details then you can refer the below link.

http://www.snailbook.com/faq/mtu-mismatch.auto.html

Cacti web access redirect from http to https

By default cacti can access using both http and https traffic. But due to the security reasons it should recommended to access cacti via https. Therefore you should change the index.php which is located in /usr/share/cacti folder.

So here is the tip on how to deploy in your Linux Machine:

1. Edit the index.php located on cacti's root folder

#cd /usr/share/cacti #vi index.php

copy the PHP code below and insert it on the first line of index.php

<?php if ($_SERVER['SERVER_PORT']!=443) { $url = "https://". $_SERVER['SERVER_NAME'] . ":443".$_SERVER['REQUEST_URI']; header("Location: $url"); } ?>

Then restart the apache server.
Open your cacti using http://your-server-ip/cacti, this will redirect you to https://your-server-ip/cacti and ask you to verify and confirm web certificate on the first use. Just proceed and complete the process and you are good to go.

Secure connection failed accessing Apache 2.4 web server by using Windows 7 machine

Error : When Windows 7 user try to access web site running on Apache 2.4 Web server it produce SSL Secure connection failure error message on the browser.

I have logged into the server and then check the ssl.conf file. There it found the below configuration in the file.

# SSL Protocol support:

# List the enable protocol levels with which clients will be able to

# connect. Disable SSLv2 access by default:

#SSLProtocol all -SSLv2

SSLProtocol all -SSLv2 -SSLv3

It has by default disabled the SSL v2 and SSL v3 support. Then I have enable the SSL v3 support by editing Above line. It's seems like this after the changes.

# SSL Protocol support:

# List the enable protocol levels with which clients will be able to

# connect. Disable SSLv2 access by default:

SSLProtocol all -SSLv2

#SSLProtocol all -SSLv2 -SSLv3

With the changes then restart the apache server. It has solved the problem.

VOIP server migration issues

I have migrated FreePBX 2.10 32 bit server to FreePBX 2.11 64 bit server. Then restored the backup. But after the reboot SIP does not response properly to its clients.

Check the asterisk logs and found some warning messages. It state that some modules can not be found.

loader.c: Error loading module 'pbx_config.so': /usr/lib/asterisk/modules/pbx_config.so: cannot open shared object file: No such file or directory

loader.c: Error loading module 'chan_local.so': /usr/lib/asterisk/modules/chan_local.so: cannot open shared object file: No such file or directory

loader.c: Error loading module 'res_mwi_blf.so': /usr/lib/asterisk/modules/res_mwi_blf.so: cannot open shared objec t file: No such file or directory loader.c: Error loading module 'pbx_config.so': /usr/lib/asterisk/modules/pbx_config.so: cannot open shared object file: No such file or directory

I have found that those modules are in the /usr/lib64/asterisk/ modules directory. Therefore I need to change the asterisk module directory path. It has located in /etc/asterisk/asterisk.conf file. Once the asterisk.conf file changed reboot the server. Then above errors are gone and server has responded to its clients properly.

The second issue was mysql connection error for CDR reports. I have checked the cdr configuration file for mysql. It is in /etc/asterisk/cdr_mysql.conf and it used previous server credentails. The new mysql user details can be taken from /etc/freepbx.conf and then it just a matter of copy and paste correct credentails to cdr_mysql.conf file.

Once you have migrated please keep continuous monitoring for at least one day. Cheers......