by Tom Millard

Install PHP 7 on Mac with homebrew

php - php7 - homebrew - mac

Working on a mac and coding with PHP?

Its probably time to install PHP 7. It's like the second coming of christ in PHP, so why not join in.

Lets start by updating, brew:

brew update;

Un-install old version of php:

brew unlink php53 php54 php55 php56;

Then will need to install php7

brew tap homebrew/dupes;
brew tap homebrew/versions;
brew tap homebrew/homebrew-php;
brew install php70;

Install composer

brew install composer;

Run it in one command, just copy and paste this into your terminal:

wget -O - https://gist.githubusercontent.com/Tom-Millard/7391a827ddfb617a4a69dbfd985a4b65/raw/8d25a0d52938b0a94858b7f63343e372076bb5e2/gistfile1.sh | bash
by Tom Millard

Spin up multiple VM's at once, with Vagrant

puppet - vagrant - devops

I recently wanted to spin up multiple VM in order to test my puppet scripts in a similar enviroment to live.

The simplest way to set up multiple VM is with vagrant.

This post will require a basic knowledge vagrant and the terminal.

First we put together a simple YAML file outlining the settings for each box:

boxes:
  -
    host: '192.168.33.142'
    name: 'node-ns'
    private_name: 'node-ns.local.puppet.dev'
  -
    host: '192.168.33.144'
    name: 'node-lb'
    private_name: 'node-lb.local.puppet.dev'
  -
    host: '192.168.33.145'
    name: 'node-a'
    private_name: 'node-a.local.puppet.dev'
  -
    host: '192.168.33.146'
    name: 'node-b'
    private_name: 'node-b.local.puppet.dev'
  -
    host: '192.168.33.143'
    name: 'node-master'
    private_name: 'node-master.local.puppet.dev'
    memory: 1024
    cpu: 2

This is a pretty basic set up, in each case i declare the ip address, name and private name of each box. With these details we can set up a FQDN server (node-ns) and have our puppet master send out manifests (node-master). I also have a node for load balancing (node-lb) between node-a and node-b.

My puppet server also has additional values in order to beef up its resources.

In our vagrant file we iterate through this YAML file and building each box:

Vagrant.configure(2) do |config|
  conf = YAML.load_file("./boxes.yaml")
  conf['boxes'].each do |box|
    config.vm.define box['name'] do |node|
      node.vm.hostname = box['private_name']
      node.vm.network "private_network", ip: box["host"]
      node.vm.box = "[point to your box]"
      node.vm.synced_folder './shared', "/shared", :nfs => { :mount_options => ["dmode=777","fmode=666"] }
      node.ssh.pty = true

      memory = if box['memory'].nil? then 256 else box['memory'] end
      cpu = if box['cpu'].nil? then 1 else box['cpu'] end

      node.vm.provider :virtualbox do |vb|
        vb.name = box['name']
        vb.memory = memory
        vb.cpus = cpu
      end

      script = "sudo sh /shared/code/sh/common.sh; \n"
      node.vm.provision "shell", inline: script, run: "always"

    end
  end
end

Each box can run its own shell scripts these shell scripts need to be accessible inside the VM, to do this all shell files are placed in a shared dir. To make it simple place everything in shared, this way accessing resources on each box is easy.

Basic directory/file structure

  • ./Vagrantfile
  • ./boxes.yaml
  • ./shared
  • ./shared/code/sh
  • ./shared/code/sh/common.sh
  • ./shared/templates
  • ./shared/templates/ntp.conf
  • ./shared/puppet/..

Each box will run a common shell script used to install some basic packages along with puppet.

My common file looks like this:

#!/bin/bash

##
#   Install bind functions
##
sudo yum -y install bind bind-utils wget;

##
#    Set time zone to Europe, this is so puppet stays up to date with all its nodes
##
sudo timedatectl set-timezone Europe/Dublin;

##
#   Install ntp
#
sudo yum -y install ntp;

##
#  Set ntpdata url so we can look up time zones
##
sudo ntpdate pool.ntp.org;

##
#  Copy over the ntp name servers so we know where to look
##
sudo cp /shared/templates/ntp.conf /etc/ntp.conf;

##
#  Restart ntpd and enable it on start-up
##
sudo systemctl restart ntpd;
sudo systemctl enable ntpd;

##
#  Install puppet agent
##
sudo rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm;
sudo yum -y install puppet;

##
#  Start the puppet agent
##
sudo -i /opt/puppetlabs/bin/puppet resource service puppet ensure=running enable=true;

(Note: I will cover setting up a private dns server, automatically in the next blog post. Digital Ocean cover setting it up manually here.)

Once each box is set up and talking to each other, we can run vagrant commands as normal. To target an individual boxes you will need to pass in the name of the box as the 2nd argument:

  • vagrant ssh node-master
  • vagrant up node-master
  • vagrant destroy node-master

Running vagrant up or vagrant destroy will run on all boxes in your enviroment.

by Tom Millard

Trouble shooting and fixing composer on linux (When it's just not working)

php - composer - linux

Composer is a php package manager used in all modern php frameworks and projects. Installing it and getting it up and running is often a simple and easy to do. However sometimes it can cause an issue.

This method is a sure fired way to get composer running globally using puppet on a Centos box (i'm sure this method can be used on any distro, not tested).

The plan is to hardwire every user to have access to composer based on where it is installed on the system. To do this will be installing composer then making an alias file which points to the composer.phar file.

First make a directory and download composer:

$ mkdir -p /usr/local/bin/composer;
$ cd /usr/local/bin/composer && wget https://getcomposer.org/composer.phar -O composer.phar';
$ chmod 777 /usr/local/bin/composer/composer.phar;

Simple, if we type the following php /usr/local/bin/composer/composer.phar install in a dir with a composer.json file we will see all our package being installed.

This is a work around, what we actually want to happen is to just type composer install. Like any easy to use system.

By adding the alias file to /etc/profile.d/ every user inherits the alias.

So will add this short-cut:

echo "alias composer='php /usr/local/bin/composer/composer.phar'" > /etc/profile.d/00-composer-alias.sh

The 00 means its parsed first.

Remember to restart your session.

This is a long work around, but a solid one. If you can get composer working right off then well done. If your finding it a pain to access globally on your distro then this method should work.

by Tom Millard

Installing PHP7 on Enterprise Linux and Fedora using puppet.

php - php7 - puppet - linux

PHP 7 is now stable and totally a thing, so all your future php projects should be using PHP 7 as standard.

At the moment installing PHP 7 using puppet is not supported by allot of the more popular puppet packages.

However the process is easy to do.

class php7::install inherits php7
{

    package { 'epel-release-7-8.noarch':
        ensure => 'installed',
        provider => 'rpm',
        source => 'https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm',
        install_options => ['-U', '-v', '-h']
    }

    package { 'webtatic-release':
        ensure => 'installed',
        provider => 'rpm',
        source => 'https://mirror.webtatic.com/yum/el7/webtatic-release.rpm',
        install_options => ['-U', '-v', '-h']
    }

    package { 'php70w':
      ensure => 'installed',
    }

    each($php7::package_ensured) |String $pack| {
      package { $pack :
        ensure => 'installed',
      }
    }

}

All we do is add the relative packages needed to install php 7 then install php 7 itself.

Of course we will likely need some additional php packages, likely php70w-fpm, php70w-common, etc so the option is there to add any additional packages, like so:

class { 'php7': package_ensured => ['php70w-fpm', 'php70w-cli', 'php70w-common', 'php70w-mysql', 'php70w-opcache', 'php70w-pdo', 'php70w-xml'] }

And here it is all wrapped up in a nice little git-repo.

by Tom Millard

Level up your workflow with Vagrant

vagrant - puppet - workflow - ruby

Vagrant is a powerful tool to help keep your development enviroment as close to your production server as possible. No matter the OS, Vagrant can help remove deployment woes.

However one of the biggest issues with vagrant is creating a development box at the beginning of every project. So lets remove that barrier and get your box up and running in a few min, and not a few hours.

(I covered how to set up a basic LAMP stack with puppet and vagrant in a previous post. More technical information can be found here)

We need to create a git repo with your new Vagrant box setting. This way we can quickly pull in the repo when we start a new project. We can separate out the box settings from the actual codebase. By putting our Vagrant box changes in a git repo we can also make changes which we can then distribute across all projects.

My basic vagrant project layout is the following:

  • maifest/default.pp
  • www/index.php
  • Vagrantfile

The manifest contains our puppet settings for this box. The index page is the default virtual dir for our web server, and of course we have our Vagrantfile.

We also have another file which is specific to the project and not the vagrant repo, so this need to go into your gitignore file:

.vagrant
config.yaml

What is our config.yaml file?

The config.yaml file contains all the setting for this project and at its simplest form it will contain:

  • name: The name of the box
  • host_name: The host name
  • network: The network ip address

The config.yaml file needs to be stored with the codebase and not the vagrant repo. So in your code base you would have the following in your gitignore:

./vagrant
!./vagrant/config.yaml

Once our file structure is all set up we need to load the yaml file into the Vagrantfile.

const = YAML.load_file('./config.yaml')
config.vm.hostname = const['host_name']
config.vm.box = "[url to our box]"
config.vm.network "private_network", ip: const["network"]
config.vm.provider "virtualbox" do |v|
    v.name = const['name']
end
  config.vm.synced_folder "../", const["vagrant_file"], :nfs => { :mount_options => ["dmode=777","fmode=666"] }

This is our vagrant file, it inherits all the values in our config.yaml file. We also reference the parent dir in the sync folder. This is the location of our codebase.

As a git repo all we need to do is run:

git clone [vagrant-repo] vagrant;
cd vagrant && vagrant up;
by Tom Millard

Install node with Puppet, on Enterprise Linux and Fedora

puppet - node

Installing Node 6 and ensuring its up and running is nice and simple with this little puppet script.

package { 'gcc-c++':
  ensure => 'installed',
}

package { 'make':
  ensure => 'installed',
}

exec { 'setup-node':
  command => 'curl -sL https://rpm.nodesource.com/setup_6.x | sudo -E bash -',
  require => [Package['curl'], Package['gcc-c++'], Package['make']],
  path   => '/usr/bin:/usr/sbin:/bin',
  unless => 'ls /usr/bin/ | grep node'
}

package { 'nodejs':
  ensure => 'installed',
}

First we install some needed packages. Every time out puppet master pings one of our puppet nodes package ensures that this package is installed and working.

exec runs and execution and gets the Node 6 setup files, but only if curl, gcc-c++ and make are installed. We avoid running this every time we ping our puppet node by checking if the Node package is installed in the usr/bin dir.

Finally we install the nodejs package with our package function.

All this together insures that node is installed and working on our servers.

by Tom Millard

Creating a custom LAMP stack Vagrant box running Centos

##The problem

Being a developer can be hard.

To enforce this statement I recently spent the last couple of days debugging an "off-the-shelf" vagrant box which was packaged about a years ago.

At the time I generated the box from https://puphpet.com/. Puphpet is still an excellent place to create a vagrant box with the added benefit of not needing to know how anything works. Unfortunately Redhat recently made some significant changes which has resulted in our box failing (see above tweet).

Why did it not fail sooner? simple: our vagrant boxes are cached locally. We had the joy of not needing to update anything due to the application we worked on being about 5 years old. So... ignorance is bliss.

It was only ever meant to be a temporary solution to our local development problem and the plan was always to build our own vagrant box which more accurately represented our productions enviroment.

##The Solution First thing was to strip out everything and start again. As versatile as Puphpet can be, we did not need everything it came with. We also didn't need half the security packages.

All we needed to build was a local LAMP stack, so we can develop with ease on our local systems.

The OS of choice had to be CentOS. CentOS is one of the more enterprise level linux distros and its what we run in production.

...we got hit with a wave of 404 mirror errors

The problems all started because we were using Centos 6.5 which is now deprecated(?) and vagrant wished to update those boxes. In doing so we got hit with a wave of 404 mirror errors followed by some puppet failures due to a difference in the puppet version. Long story short it was a mess. A decision was made to re-build the box and simplify the process.

We decided to role out Centos 7. The latest stable version. Our production enviroment is only on 6.8 however; but we might be rolling out a update to this. #Security

(Read future plans at the end for why this will change)

###Keeping the box

Normally we pull our box down from the vagrant website. However this was not working out and they also added some strange login wall?

I got a fresh copy from Redhat. I then took that image and uploaded it to our local webserver.

###Failure to mount drive

After this I made a very simple vagrant file with just a reference to my new local box.

$vagrant up

So that worked.. kind of. The vagrant file failed to mount to the vagrant directory and so I was unable to modify files between my local OS and the virtual box.

Solution: You need to specify in the Vagrant file that you want this to happen. Bit of a strange one as the normal default response is: yes i want to mount the vagrant file.

config.vm.synced_folder "./", "/vagrant"

This will map the directories to its default location.

Also worth noting is: do not add anything to this line, half the additional arguments fail. Just leave the default group access and directory type.

Problem number 2: SELinux

By default Centos comes with SELinux turned on. SELinux is a must on your production enviroment. However on a development enviroment it restricts us; and one area it restricts us the most is when it comes to mounting our vagrant file. So lets just turn it off.

SELinux can be disabled here: /etc/selinux/config. SSH into your vagrant box and then use vi to go into this file and change it to look like this.

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

In order for this to take effect the box needs to be rebooted.

Rebooting the box will interfere with our set-up so we need to re-package the box with SELinux turned off. Exit your vagrant box and repackage what we have so far:

vagrant package [name]

Optionally upload this box to your local webserver and reference the new box in your Vagrant file.

##Using Puppet

I decided to keep with puppet instead of building a box then re-packaging it. The decision to use puppet means that we can be more flexible with our system set up in the future without having to re-build the box again.

###Avoiding the default Vagrant config

Vagrant comes with allot of configuration option you can apply. However some of them were causing errors with our new box so I made the decision to avoid them and instead make all the necessary configurations to the box with shell scripts and puppet packages.

This means I relied heavily on the inline shell argument.

  config.vm.provision :shell do |shell|
    shell.inline = "[all the things]"
  end

Another reason for doing this is the lack of puppet on the new box. In order to get puppet up and running you will need to add the following commands to your shell variable:

  config.vm.provision :shell do |shell|
    shell.inline = "rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm;
                    yes | yum -y install puppet;
                    "
  end

Once the box is built the shell commands will be run. Above puppet is downloaded and installed. Along with wget.

(The argument -y means: Answer yes to everything)

###Downloading puppet packages

To avoid too much bloat in the git repo I decided to pull down the puppet packages I was using on the fly. So I added the following lines to the shell argument.

                    sudo touch /etc/puppet/hiera.yaml;
                    sudo rm -rf /etc/puppet/modules/*;
                    mkdir -p /etc/puppet/modules;
                    puppet module install thias-php;
                    puppet module install puppetlabs-mysql;
                    puppet module install puppetlabs-apache;

The first line creates a yaml file which stops puppet throwing a warning. After this we remove any packages from the modules file (just incase), then we make it again. Finally we pull down php, mysql and apache puppet packages.

(Read future plans at the end for why this will change)

###Building the box

After we pull down the packages we will need to make a reference to the puppet manifest file. My manifest file is in the vagrant dir under - manifest.

sudo puppet apply /vagrant/manifests/default.pp;

This will then run our manifest file.

I will post up a simple default.pp file after this post. However the puppet modules have very good documentation and it shouldn't take long to set up (copy and paste is your friend).

###Altogether

You should end up with something like this:

Vagrant.configure(2) do |config|
  config.vm.hostname = "local.dev"
  config.vm.box = "http://192.168.1.1/boxes/centos-7.box"
  config.vm.network "private_network", ip: "192.168.33.135"
  config.vm.synced_folder "./", "/vagrant"

  config.ssh.pty = true

  config.vm.provision :shell do |shell|
    shell.inline = "rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm;
                    yes | yum -y install puppet;
                    sudo yum -y install wget;
                    sudo touch /etc/puppet/hiera.yaml;
                    sudo rm -rf /etc/puppet/modules/*;
                    mkdir -p /etc/puppet/modules;
                    puppet module install thias-php;
                    puppet module install puppetlabs-mysql;
                    puppet module install puppetlabs-apache;
                    sudo puppet apply /efiling/vagrant/manifests/default.pp;
                    "
  end

end

##Other Configurations

Normally once this is done you might find your missing some PHP packages or you need to import an sql file into your new database. This can all be done using shell scripts then simply add a line in to run each one.

sh /vagrant/scripts/mysql-setup.sh

##Future plans

Although I said it was a bad idea to keep the puppet packages under version control, you should still keep a local version. If the author of the packages changes something; or they simply delete the repo you will have to debug your box again. So keep a copy of the puppet packages. Probably in the same place as your box is and download them with wget when your need them.

wget http://192.168.1.1/packages/thias-php

Running a different version of your production OS is going to lead to trouble. One of the issues I encountered was, CentOS dropping support for mysql and favouring MariaDB. The mysql puppet package i use instals MariaDB but our production enviroment uses mysql. I have yet to find an issue but its not something I want to test for very long.

Next job is to drop CentOS 7 and package up a copy of CentOS 6.7 which will have support for mysql and will be faithful to our production enviroment.

Another issue is a heavy reliance on shell scripts. Although in this situation its not a problem it would be better to move everything over to puppet.

by Tom Millard

Running SQL queries in Codeception through PDO

Codeception provides some basic functions for testing the current state of your database. However when you need to run more complex queries you will need to expose the Db module.

This function will need adding to the relevant helper class and is then available in your tests, normally though $I .

public function runSqlQuery($query){
  $dbh = $this->getModule("Db")->dbh;
  $result = $dbh->query($query);
  return $result->fetchAll();
}

I added this to my Helper/Acceptance class which allowed me to check the state of the app in my Cept tests.

by Tom Millard

Bind events on dynamic content using jQuery

javascript - jquery

When you inject a view into a container any jQuery click event within that view will need to be bound again.

The best way to bind actions to dynamic content when using jQuery is to listen out for new DOM elements and bind on them when they appear.

$('#somecontainer').on('click' ,'.js-tool-tip', actionListenedToolTip);

In this piece of code any html injected into '#somecontainer' which has the class '.js-tool-tip' will have our click event bound to it.

by Tom Millard

Generate a set of width's with a sass loop

css - sass

For large scale applications with dynamic content you will likely need a set of css tools. I normally have a _tools.scss file where I place generic pieces of code for general purpose use.

If you need a set of out the box width, this piece of code will create you a set of 10% width which you can use for any purpose.

@for $i from 1 through 10 {
  .w--#{ ($i * 10) } { width : #{ ($i * 10) }% }
}

You could also use this idea to put together a series of light-weight css styles to create a micro css framework, like this - basscss