Get MySQL Tabel size sorted

Recently to dump a database with lots of large table which are not needed, I had to find out the large tables first and then use sequel pro to export the database with small tables and only structure of large table. I have used the following query to find out the large tables:

SELECT 
     table_schema as `Database`, 
     table_name AS `Table`, 
     round(((data_length + index_length) / 1024 / 1024), 2) `Size in MB` 
FROM information_schema.TABLES 
ORDER BY (data_length + index_length) DESC;
Advertisements

Enable Content Compression in nginx

To enable content encryption in nginx, put the below configuration inside server block of nginx configuration file (usually in /etc/nginx/nginx.conf):

       gzip on;
       gzip_disable "msie6";
       gzip_http_version  1.1;
       gzip_comp_level    5;
       gzip_min_length    256;
       gzip_proxied       any;
       gzip_vary          on;
       gzip_types
           application/atom+xml
           application/javascript
           application/x-javascript
           application/json
           application/rss+xml
           application/vnd.ms-fontobject
           application/x-font-ttf
           application/x-web-app-manifest+json
           application/xhtml+xml
           application/xml
           font/opentype
           image/svg+xml
           image/x-icon
           text/css
           text/plain
           text/x-component;

You should get content size reduction unto 80%!

Upgrade PHP version to 5.5 in amazon linux

If you have older version of php in your amazon linux instance, follow the below steps to upgrade it to php 5.5:

  1. Find the list of old php packages and remove those:
    sudo yum list installed | grep "php"
    
    sudo yum remove php php-cli php-common php-devel php-fpm php-gd php-imap php-mbstring php-mcrypt php-mysql php-odbc php-pdo php-pear php-pecl-apc php-process php-xml (adjust according to your list from previous command)
    
    sudo yum remove httpd*
  2. Install new packages
  3. sudo yum install php55 php55-devel php55-common php55-cli php55-pecl-apc php55-pdo php55-mysql php55-xml php55-gd php55-mbstring php-pear php55-mcrypt mysql55 php55-server php55-mysqlnd php55-fpm
    
  4. Restart servers
  5. sudo service php-fpm restart
    sudo service nginx start
    sudo service mysqld start
    
  6. Check php-fpm config:
    sudo vim /etc/php-fpm-5.5.d/www.conf
    <blockquote> Enter the following content:
    listen = /var/run/php-fpm/php-fpm.sock
    listen.owner = nginx
    listen.group = nginx
    listen.mode = 0664
    
    user = nginx
    group = nginx
    

PHP Conference Asia 2015 Recap

The First Pan-Asian PHP conference happened in Singapore on last 22 – 23 September, 2015. Many international and regional speakers presented their valuable speeches in the event.

Group Picture from PHPConfAsia

Group Picture from PHPConfAsia

While in USA, EU lots of community events happens through the year, comparatively very few events happen in Asia! Now at least some people has taken the initiative to organize this great event. Heartiest thanks to Michael Cheng (@coderkungfu) and other people who worked hard to make the conference smooth & enjoyable.
It was really an awesome experience being in the conference.

At Widespace, we use PHP (specifically Laravel) for several platforms, two of us from our Dhaka office joined the event.

Events on Tuesday:

  • Rasmus Lerdorf started the conference with his keynote presentation. It was mostly about upcoming PHP 7. Listening about the next PHP version from the PHP creator was really exciting. He showed some new feature coming with PHP 7, compatibility and also provided performance comparison of PHP7 with other PHP versions and HHVM for different projects. You can get the rich presentation slides here.
  • After the tea break, Stéphane Boisvert came into stage with his in-depth discussion about WordPress Security. He discussed different security topics like sql injection, csrf attack, sanitization, authorization in wordpress. Thanks Stéphane for providing nice ‘WordPress’ branded Sunglasses 🙂 His talk will be very valuable for WordPress developers.
  • Later came a storyteller, telling a faily tale with animation! Well, Steven Cooper described PayPal and Braintree API’s and how they can be integrated into code with the help of fairy tale. You can enjoy his fairy tale here.
  • Before Lunch, the last session was about application (Magento) performance improvement by Harald Zeitlhofer from Austria. He gave an example how a magento installation performance issues were detected and corrected with the help of Dynatrace performance monitoring tool.
  • First session after awesome lunch was by Jack Lenox. He has shown the way of combining React development techniques with a more traditional PHP workflow. His lecture can be found here.
  • After that Bagus Aji Santoso came into stage telling about Panada Framework, Simple but High Performance PHP Framework from Indonesia. He was a bit shy, presenting for the first time in front of international audience 🙂 Panada framework seems to be popular in Indonesia powering up several popular local sites. See features about Panada in presentation here.
  • Edwin Ong then gave a lightning talk about PHP streams. PHP Streams were introduced with PHP 4.3.0 as a way of generalizing file, network, data compression, and other operations which share a common set of functions and uses. He discussed how we use it knowingly or unknowingly regularly.
  • Before the tea break, last talk was given by Sam Yong, a student from NUS, with topic: Go Reinvent The Wheel. It is certainly true that reinventing the wheel in software industry drives numerous innovations which we see today.
  • Pierre Joye, PHP Core Developer and OSS contributor, then gave rich talk about PHP7, HHVM and Co. Besides showing performance statistics, new features about PHP 7, he also talked about Zephir and alternate PHP implementations including HHVM. See his presentation.
  • Premshree Pillai explained how they built a scalable system for tracking shipping packages for Etsy. He shared their architecture in Etsy and interested audience also asked several questions about their implementation. His presentation is shared in slideshare.
  • The last session of first day was with the topic ‘Why Your Test Suite Sucks’ given by Ciaran McNulty, Senior Trainer, Inviqa. He has nicely shown how to evolve from no-test to TDD state. He also showed several code snippet and how to improve the code in those cases. We should really follow the practices that Ciaran mentions in this nice presentation.

Events on day 2 will be published soon…

Performance of yield vs built-in range function

Just out of curiosity, tried to benchmark the memory used by PHP generator(yield) and range function.
Code used here:

function xrange($start, $limit, $step = 1) {
	if ($start < $limit) {
		if ($step <= 0) {
			throw new LogicException('Step must be positive');
		}

		for ($i = $start; $i <= $limit; $i += $step) { yield $i; } } else { if ($step >= 0) {
			throw new LogicException('Step must be negative');
		}

		for ($i = $start; $i >= $limit; $i += $step) {
			yield $i;
		}
	}
}

$preMemory = memory_get_usage(true);
echo 'Before: ' . $preMemory/1024;

$total = 0;
foreach(xrange(1, 1000000) as $i) {
	$total += $i;
}

echo "\nYield Took: " . (memory_get_usage(true) - (float)$preMemory)/1024;
$preMemory = memory_get_usage(true);

$total = 0;
foreach(range(1, 1000000) as $i) {
	$total += $i;
}

echo "\nrange() Took: " . (memory_get_usage(true) - (float)$preMemory)/1024;

Result is (in kilobyte):

Before: 256
Yield Took: 0
range() Took: 768

Increasing the loop 10 times, makes the range() cause stack overflow!

Resolving GuestAdditions version mismatch in vagrant/homestead vm (failed to mount shared folders / modprobe vboxsf failed)

While trying to install homestead machine for Laravel, got some annoying issue: failed to mount shared folders. I had vagrant installed earlier with different version than homestead, causing the problem.
The output from ‘homestead up’ was:

$ homestead up

/opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/pre-rubygems.rb:31: warning: Insecure world writable dir /usr/local in PATH, mode 040777
/opt/vagrant/embedded/gems/gems/bundler-1.10.5/lib/bundler/shared_helpers.rb:78: warning: Insecure world writable dir /usr/local in PATH, mode 040777
Bringing machine 'default' up with 'virtualbox' provider...
==&gt; default: Checking if box 'laravel/homestead' is up to date...
==&gt; default: Clearing any previously set forwarded ports...
==&gt; default: Clearing any previously set network interfaces...
==&gt; default: Preparing network interfaces based on configuration...
default: Adapter 1: nat
default: Adapter 2: hostonly
==&gt; default: Forwarding ports...
default: 443 =&gt; 44300 (adapter 1)
default: 3306 =&gt; 33060 (adapter 1)
default: 5432 =&gt; 54320 (adapter 1)
default: 80 =&gt; 8080 (adapter 1)
default: 81 =&gt; 8181 (adapter 1)
default: 86 =&gt; 8686 (adapter 1)
default: 82 =&gt; 8282 (adapter 1)
default: 22 =&gt; 2222 (adapter 1)
==&gt; default: Running 'pre-boot' VM customizations...
==&gt; default: Booting VM...
==&gt; default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
default: Warning: Connection timeout. Retrying...
==&gt; default: Machine booted and ready!
Got different reports about installed GuestAdditions version:
Virtualbox on your host claims: 5.0.4
VBoxService inside the vm claims: 4.3.6
Going on, assuming VBoxService is correct...
GuestAdditions seems to be installed (4.3.6) correctly, but not running.
Got different reports about installed GuestAdditions version:
Virtualbox on your host claims: 5.0.4
VBoxService inside the vm claims: 4.3.6
Going on, assuming VBoxService is correct...
Starting the VirtualBox Guest Additions ...fail!
(modprobe vboxsf failed)
Got different reports about installed GuestAdditions version:
Virtualbox on your host claims: 5.0.4
VBoxService inside the vm claims: 4.3.6
Going on, assuming VBoxService is correct...
Removing existing VirtualBox DKMS kernel modules ...done.
Removing existing VirtualBox non-DKMS kernel modules ...done.
Building the VirtualBox Guest Additions kernel modules
The headers for the current running kernel were not found. If the following
module compilation fails then this could be the reason.

Building the main Guest Additions module ...done.
Building the shared folder support module ...fail!
(Look at /var/log/vboxadd-install.log to find out what went wrong)
Doing non-kernel setup of the Guest Additions ...done.
Restarting VM to apply changes...
==&gt; default: Attempting graceful shutdown of VM...
==&gt; default: Booting VM...
==&gt; default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
default: Warning: Connection timeout. Retrying...
==&gt; default: Machine booted and ready!
==&gt; default: Checking for guest additions in VM...
default: The guest additions on this VM do not match the installed version of
default: VirtualBox! In most cases this is fine, but in rare cases it can
default: prevent things such as shared folders from working properly. If you see
default: shared folder errors, please make sure the guest additions within the
default: virtual machine match the version of VirtualBox you have installed on
default: your host and reload your VM.
default:
default: Guest Additions Version: 5.0.4
default: VirtualBox Version: 4.3
==&gt; default: Setting hostname...
==&gt; default: Configuring and enabling network interfaces...
==&gt; default: Mounting shared folders...
default: /vagrant =&gt; /Users/shafiul/source/ws_ui_vagrant
Failed to mount folders in Linux guest. This is usually because
the "vboxsf" file system is not available. Please verify that
the guest additions are properly installed in the guest and
can work properly. The command attempted was:

mount -t vboxsf -o uid=`id -u vagrant`,gid=`getent group vagrant | cut -d: -f3`, vagrant /vagrant
mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant`, vagrant /vagrant

The error output from the last command was:

/sbin/mount.vboxsf: mounting failed with the error: No such device

Digging into the error message, found clue in below lines:

Virtualbox on your host claims:   5.0.4
VBoxService inside the vm claims: 4.3.6

So, I decided to install VBoxGuestAdditions 5.0.4 in both host and guest.
In some cases, it has been observed that only running the below command solves the problem:

$ vagrant plugin install vagrant-vbguest

and then run homestead up again.

In case, the error persists, then follow the below steps, which I had to follow to run my homestead machine.

Though the mounting of shared folder failed while starting vm, but vm started successfully. I needed to ssh into vm by homestead ssh and run the following commands inside vm to install dependencies for VBoxGuestAdditions:

$ sudo apt-get update && sudo apt-get install linux-generic
$ sudo apt-get install dkms build-essential linux-headers-$(uname -r)
$ sudo apt-get install binutils gcc make patch libgomp glibc-headers glibc-devel kernel-headers
$ sudo apt-get install kernel-devel
$ sudo apt-get install kernel-pae-devel
$ sudo apt-get install virtualbox-guest-dkms

$ sudo apt-get install xserver-xorg xserver-xorg-core

I am not sure if the last line was needed, please try without it, and see if you get error in next step.

Inside the vm:

$ cd /opt
$ ls 
VBoxGuestAdditions-4.3.6  VBoxGuestAdditions-5.0.4

but the later was empty. So I needed to download and install it.

$ cd /opt/
$ sudo wget http://download.virtualbox.org/virtualbox/5.0.4/VBoxGuestAdditions_5.0.4.iso
$ sudo mount VBoxGuestAdditions_5.0.4.iso -o loop /mnt
$ sudo sh VBoxLinuxAdditions.run
Output should be like below:
Verifying archive integrity... All good.
Uncompressing VirtualBox 5.0.4 Guest Additions for Linux............
VirtualBox Guest Additions installer
Removing installed version 4.3.6 of VirtualBox Guest Additions...
Copying additional installer modules ...
Installing additional modules ...
Removing existing VirtualBox DKMS kernel modules ...done.
Removing existing VirtualBox non-DKMS kernel modules ...done.
Building the VirtualBox Guest Additions kernel modules ...done.
Doing non-kernel setup of the Guest Additions ...done.
You should restart your guest to make sure the new modules are actually used

$ cd /opt/
$ sudo umount /mnt
$ rm -rf *.iso

In my host mac(yosemite), I had to install the new GuestAdditions

$ sudo wget http://download.virtualbox.org/virtualbox/5.0.4/VBoxGuestAdditions_5.0.4.iso
$ sudo mv /Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso ~/.homestead
$ sudo cp VBoxGuestAdditions_5.0.4.iso /Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso

You can remove VBoxGuestAdditions.iso in ~/.homestead dir.

Next step is updating the vagrant version to 5.0.4. Link to installer: http://download.virtualbox.org/virtualbox/5.0.4/VirtualBox-5.0.4-102546-OSX.dmg

And then finally homestead machine ran successfully.