Goodbye Wordpress

My blog has been running on the Wordpress platform for years. During the last year my development focus has been partly learning javascript and mobile development. As such, the time has come to put that knowledge to use in my personal projects. In the last couple of weeks I have migrated the old Wordpress platform to a new platform: Ghost) It is pure javascript and is more focussed on pure publishing than Wordpress. What do you think?

Running CI end-to-end testing on Sauce Labs with a React hybrid cordova app

In the wonderful world of crossplatform App development it’s hard to do end-to-end testing and CI. We have been able to setup CI for the CHECK web application, and I wanted to use the experience to make it work for mobile as well.

Those React mobile apps generally run in a Cordova container, not as a native app. Of course there is the React Native project, which will run directly through a javascript bridge, but for now we are still using Cordova.

Today I will show you from scatch how we go about this.

This delivery process consists of the following steps:

  • Part 1
    • building the React application (with flux as example)
  • Part 2
    • embed the application in a Cordova container
    • prepare the container
    • ~~build the Cordova container into a package that ~~
  • Part 2 (Android)
    • embed the application in a Cordova container
    • prepare the container
    • build the Cordova container into a package that can run in the Genymotion Android emulator
  • Part 2 (iOS)
    • embed the application in a Cordova container
    • prepare the container
    • build the Cordova container into a package that can run in the iOS simulator
  • Part 3
    • upload the app package to Sauce labs
    • create the Appium test (with Mocha)
    • run the test on Sauce Labs

About me

So now you’re reading this trying to find out ‘who’s this guy with this weird blog!’. So that would be me then. Let me introduce myself.

I’m married and have two kids. Started working in IT in the 90’s after a failed attempt to study Public Administration. I liked it so much, I stayed in the industry.
Started as an administrator, got to developing and now I like both worlds.

Several hobby’s, but some of them: Astronomy, Photography, Gardening….

Here is my CV (click for the whole thing):
CV

Mobile testing with Appium (and later Sauce Labs)

Building a Continuous Delivery flow was not our only goal. Getting automated testing in there would be very helpful. So the first idea was to get visual testing going using Appium. That took me a day or so, but we got it working.

Appium startup screen

The tests are very simple and for now only prove that tapping or clicking works, the assertions are not finished yet. Tests are written in ‘Mocha’ style and can be executed from the command line and as a job on our Jenkins servers later on :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
it("should contain a button for logging in ", function () {
return driver
//find the login button
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAButton[1]")
.should.eventually.exist
});

it("a succesful login should be followed by a list of projects", function () {
return driver
//click the login button
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAButton[1]").click().sleep(1000)
//the row with a project should show up
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAStaticText[10]")
.should.eventually.exist
});

it("should be able to get details of project", function () {
return driver
//click the row of the project
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAStaticText[9]").click()
//the project name field should appear
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAStaticText[3]")
.should.eventually.exist
});

it("should be able to set this project in edit mode and save state", function () {
return driver
//click the aanpassen area
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAButton[2]").click()
//click the name input field to make it editable
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIATextField[1]").click()
//wait a bit for client server communication
.sleep(5000)
//click save
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAButton[2]").click()
//wait a bit for client server comm
.sleep(5000)
//go back to main project list
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAButton[1]").click()
.sleep(5000)
//project rows should be there
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAStaticText[10]")
.should.eventually.exist
});

it("should be able logout", function () {
return driver
//click the logoutbutton
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAButton[1]").click()
//.sleep(1000)
//click on yes
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAButton[5]").click()
.sleep(1000)
//a row with a project should show up
.elementByXPath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIAStaticText[10]").isDisplayed()
});

Below is the inspector used to find out how controls can be identified by Xpath-value.
Appium Inspector

Welcome to Ghost

You’re live! Nice. We’ve put together a little post to introduce you to the Ghost editor and get you started. You can manage your content by signing in to the admin area at <your blog URL>/ghost/. When you arrive, you can select this post from a list on the left and see a preview of it on the right. Click the little pencil icon at the top of the preview to edit this post and read the next section!

Getting Started

Ghost uses something called Markdown for writing. Essentially, it’s a shorthand way to manage your post formatting as you write!

Writing in Markdown is really easy. In the left hand panel of Ghost, you simply write as you normally would. Where appropriate, you can use shortcuts to style your content. For example, a list:

  • Item number one
  • Item number two
    • A nested item
  • A final item

or with numbers!

  1. Remember to buy some milk
  2. Drink the milk
  3. Tweet that I remembered to buy the milk, and drank it

Want to link to a source? No problem. If you paste in a URL, like http://ghost.org - it’ll automatically be linked up. But if you want to customise your anchor text, you can do that too! Here’s a link to the Ghost website. Neat.

What about Images?

Images work too! Already know the URL of the image you want to include in your article? Simply paste it in like this to make it show up:

The Ghost Logo

Not sure which image you want to use yet? That’s ok too. Leave yourself a descriptive placeholder and keep writing. Come back later and drag and drop the image in to upload:

![A bowl of bananas]

Quoting

Sometimes a link isn’t enough, you want to quote someone on what they’ve said. Perhaps you’ve started using a new blogging platform and feel the sudden urge to share their slogan? A quote might be just the way to do it!

Ghost - Just a blogging platform

Working with Code

Got a streak of geek? We’ve got you covered there, too. You can write inline <code> blocks really easily with back ticks. Want to show off something more comprehensive? 4 spaces of indentation gets you there.

.awesome-thing {
    display: block;
    width: 100%;
}

Ready for a Break?

Throw 3 or more dashes down on any new line and you’ve got yourself a fancy new divider. Aw yeah.


Advanced Usage

There’s one fantastic secret about Markdown. If you want, you can write plain old HTML and it’ll still work! Very flexible.

That should be enough to get you started. Have fun - and let us know what you think :)

2016 - the next big thing?

So now we are a day forward in the new year and I realized I’d like to say something about it. For me 2015 has been a big year: finished my studies, learning new things about javascript, cloud business, apps, Agile and DEVOPS, amazing vacationing to Portugal with my love, a couple of weeks in the U.K. with the family in the summer, getting to run 10 KM … a lot of things.

Stonehenge gathering

Thinking about the past, the future

So what will 2016 bring.. ? I hope more knowledge, challenges, love and fun and more health! All the best!

Mobile CD

This year we created Continuous Integration (CI) for CHECK, but since last month we also are in the process of creating an automated build for our mobile projects. In the DEVOPS and Agile world we call this Continuous Delivery (CD). Why would we do this? Mainly for trying to speed up delivery of applications and in shorter iterations. We noticed in the initial projects we lost a lot of time because of deployment issues, especially with all the different App Stores around.

So we tried to use our existing CI platform. This was quite a challenge, because for instance Apple devices require some specific tools on the OS X platform, and our Jenkins CI server is running on Linux. Fortunately Jenkins also works with a master-slave model, so you can setup a Mac computer as a slave to the master build server. This way we have been able to run building and signing of iOS apps on our Mac mini and storing the results in the software repository of our Configuration management system in Confluence.

Workflow diagram
Mobile Development workflow

Jenkins cookbook for CHECK

We are using Jenkins Continuous Integration Server for automating tasks that would otherwise take days and generate more errors surely. We build the database server and web infrastructure daily for our most important application (legacy with asp classic,NET 2.0/3.5 parts and a lot of Oracle 11g PL/SQL components). When completed, a set of integration tests are run with selenium.
Diagram of CHECK workflow
Deployment workflow

How we came to this

While developing on a dedicated test server for years we encountered integration problems frequently when deploying code to production. First we started using Nant to create automated deployment. This way we could guarantee deploying the software the same way every time. The real problem was that the Oracle test database was starting to differ too much from the production site. To make sure we always used the same infrastructure I took up the laborious task of creating scripts for every object in the Oracle database and put them under version control. Finally, I also created all the scripts to create the Oracle database from the command line and bundled them with the object creation scripts.By then (december 2014) we had to migrate the entire application to another infrastructure provider, so I used the new scripts to create the new environment there from scratch. The servers are all virtual, using Virtualbox technology. For orchestration I used Vagrant, the amazing tool with config files in Ruby) style by Mitchell Hashimoto, coupled with Chef (also Ruby) for further automation and provisioning when the virtual machine is up and running.In march 2015 I created nightly build jobs in Jenkins to create the database and web infrastructures.

The steps in this build:

Step 1: Deploying the database server

  • deploying a Vagrant Windows 2012 box that is somewhat preconfigured
  • install the Chef client for windows
  • start a Chef provision run
  • installing a telnet client
  • installing .Net 3.5
  • getting the database creation and configuration scripts from git
  • installing Oracle 11g
  • creating an Oracle database with several tablespaces
  • creating an Oracle Listener
  • creating Oracle schema’s (about 7)
  • create database objects in Oracle (around 500 tables and 80000 lines of PL/SQL code and many more objects)
  • populate the entire database with data
  • refresh all indexes and snapshots and statistics

Step 2: Deploying the web server

  • deploying a Vagrant Windows 2012 box that is somewhat preconfigured
  • install the Chef client for windows
  • start a Chef provision run - installing a telnet client
  • installing .Net 3.5
  • installing the IIS web server Role for Application development with ISAPI extensions, ASP.NET and such
  • mapping a drive to a data volume
  • installing an oracle client using a silent install with answer file
  • configuring the oracle client by placing configuration files (tnsnames.ora and such)
  • do a git clone of the custom asp code that runs on the server
  • open the local firewall for some web ports
  • set the timezone to CET
  • disabling NLA for rdp
  • configure IIS with a custom recipe : - creating and configuring an app pool for the application
  • create a site to host the code
  • configure the site with some debug attributes for development

Step 3: Running integration tests

  • running Selenium tests which where recorded with firefox
  • each test does: - login test
  • use case test

Step 4: Automated deployment (when the integration tests succeed)

  • creates a deployment zip package
  • populates the zip with a file structure for the Acceptance environment and production environment
  • runs a nodejs javascript to edit the config files and set them to the correct values in each environment directory structure
  • updates the README in the main directory with instructions
  • the zip package that is ready to be deployed is uploaded to our configuration management system

DEVOPS tools of the trade

One of the important concepts that came to DEVOPS from the Extreme Programming guys was Continuous Integration. It means you build and assemble together daily all the software that has been developed for your product and runs tests if it all still works.

Diagram CD
CI workflow

When someone made an error or introduced a sneaky bug that breaks the product (we call it ‘regression’), it is immediately clear and this way it can be handled before shipping.
I really like this idea (as a system engineer) and I have introduced this concept to our software some years ago. Although many tools for build automation and integration exist let me first mention the (real) big ones. I will explain which one we used in a new post.

Jenkins

A server deployed in a Tomcat servlet container for running automated builds and the plugins and workflow structure allow you to connect to Configuration management tools, testservers, name it.. You can hook it up to your desired versioning system like git or svn. For instance, you can trigger a build when a developer commits new code and build the source code and run tests afterwards (or deploy to some environment). This has been around for some time and their plugin system is quite rich. You can integrate anything with Jenkins.

Teamcity

Created by Jetbrains who also make beautiful IDE’s (IntelliJ, Webstorm). Arguably the same features as Jenkins but it seems to be more Microsoft friendly.

Bamboo

The CI system by Atlassian also delivers features for building and testing of software in a workflow. The strongpoint of this solution is that it ties in neatly with other Atlassian products like Jira and Confluence. It is based on Open Source tooling.

Team foundation Server (TFS)

Microsoft’s entry is this market. It can handle not just building and testing alone but it offers release management and projectmanagement capabilities as well. It combines nicely with all Microsoft server products (of course). I used to vote against it because I don’t like the idea of vendor lock-in, but with the new wind blowing inside Microsoft introduced by their CEO Nadella and their embrace of Open Source.. Could work in a heterogeneous environment.