Docker Swarm Sample

Docker swarm sample with NodeJS app on Hyper-V

Prerequisites


  • This sample is using Hyper-V

1. Clone sample repository

git clone https://github.com/wswijaya/docker-swarm-node-app.git

2. Build Docker Image and push to Docker Hub

docker build -t /node-app-sample .
docker push /node-app-sample

3. Create VMs

docker-machine create -d hyperv --hyperv-virtual-switch "myswitch" myvm3
docker-machine create -d hyperv --hyperv-virtual-switch "myswitch" myvm2
docker-machine create -d hyperv --hyperv-virtual-switch "myswitch" myvm3
docker-machine create -d hyperv --hyperv-virtual-switch "myswitch" myvm4

4. Create Docker Swarm and set VM1, VM3, and VM4 as manager

docker-machine ssh myvm1 "docker swarm init"
docker-machine ssh myvm1 "docker swarm join-token manager"
docker-machine ssh myvm3 "docker swarm join --token :"
docker-machine ssh myvm4 "docker swarm join --token :"
docker-machine ssh myvm1 "docker swarm join-token worker"
docker-machine ssh myvm2 "docker swarm join --token :"

5. Deploy Services

docker-machine scp docker-compose-node-app.yml myvm1:~
docker-machine ssh myvm1 "docker stack deploy -c docker-compose-node-app.yml nodeapplab"
docker-machine ssh myvm1 "docker stack ps nodeapplab"

6. Test Site

7. Remove Services

docker-machine ssh myvm1 "docker stack rm nodeapplab"

8. Remove Docker Swarm

docker-machine ssh myvm1 "docker stack rm nodeapplab"
docker-machine ssh myvm2 "docker swarm leave"
docker-machine ssh myvm3 "docker swarm leave --force"
docker-machine ssh myvm4 "docker swarm leave --force"
docker-machine ssh myvm1 "docker swarm leave --force"

Optional: Configure Load Balancer using HAProxy

Installation steps for HAProxy on Ubuntu -> How to use HAProxy

Sample config file /etc/haproxy/haproxy.cfg

global
daemon
maxconn 200

defaults
mode http
timeout connect 5000ms
timeout client 10000ms
timeout server 10000ms

frontend http-in
bind *:9090
default_backend servers

backend servers
server vm1 : maxconn 40
server vm3 : maxconn 40
server vm4 : maxconn 40

The Future of Software Engineering

Software is the invisible thread and hardware is the loom on which we weave the fabric of computing.

Great keynote by Grady Booch at ICSE 2015. This is a must watch keynote. Grady started his keynote with history of computing, cover the past, present, and future (start from 00:12:00). Following that he talked about how those changes affect us as software engineer (start from 00:46:26), then he closed his keynote with challenges in software engineering and how we can contribute to address to challenges (start from 1:10:00).

No matter what future you might imagine, it will rely on software-intensive systems not yet built.

Agile Architecture

​​​​”Agile Architecture” – Molly Dishman & Martin Fowler Keynote​

It covers:

  1. What software architecture is​
  2. Misinterpretation in agile
    • ​​​​Working software and​ no documentation
    • ​​Responding to change and having no plan​​
  3. How software architecture is done in agile projects​​​ and roles of software architect
​Personally I think ‘good’ software architecture enables agility and​ allowing teams to respond to change faster.

Successful IT Systems Demand Serious IT Governance

A few days ago I read a HBR blog on the trouble in HealthCare.gov and it is a different perspective. What is interesting here is, the author emphasizes on good governance then the technical issue.

Key highlights from the blog is “Good governance, not superior technical chops or ready access to alpha geeks, is how you build complex systems that deliver reliable and resilient value for money. Good governance provides oversight, insight, and foresight. Emerging problems are flagged sooner; project leaders present their testing protocols and outcomes; and updated expectations are clearly communicated throughout the enterprise. Contingencies are constantly reviewed and appropriately revised.”

If you are interested to read more, here is link to the blog post “Don’t Blame IT for Obamacare’s Tech Troubles“.

Deploying Grails Application to CloudFoundry

Here I am sharing the steps to deploy Grails Application to CloudFoundry.

Step 1: Install CloudFoundry Plugin

install-plugin cloud-foundry

Step 2: Configure Credential
Create $HOME/.grails/settings.groovy and configure your CloudFoundry credential there. Putting your credential in settings.groovy makes it available to all your Grails projects. The alternative is to configure in BuildConfig.groovy or Config.groovy which makes it available to that specific project only. Using settings.groovy is better as it keep the credential out of source control. In Windows, $HOME= C:\Users\<username>.

grails.plugin.cloudfoundry.username = ''
grails.plugin.cloudfoundry.password = ''

Step 3: Verify Credential
Check whether the credential is working by executing the following command:

cf-info

You will see a similar print out as shown below

VMware's Cloud Application Platform
For support visit support@cloudfoundry.com
Target: http://api.cloudfoundry.com (v0.999)
User:     <your_username>
Usage:    Memory   (128.0M of 2.0G total)
Services (0 of 16 total)
Apps     (1 of 20 total)

Step 4: Deploy Grails Application
Now, deploy your Grails application by executing the following command:

cf-push

If you decided to deploy your application with a different name, use the option –appname

cf-push --appname=other-name

During deployment you will be prompted with the following questions. The mysql service can be provisioned at later stage using the cf-create-service.

You're running in the development environment but haven't specified a war file, so one will be built with development settings. Are you sure you want to do proceed? ([y], n)
Application Deployed URL: 'grails-qotd-extjs.cloudfoundry.com'?
Would you like to create and bind a mysql service? ([y], n)

Step 5: Verify Application Deployment
To check your deployment, execute the command below:

cf-apps

You will see a similar print out as shown below

+-----------------+----+---------+----------------------------------+----------+
| Application     | #  | Health  | URLs                             | Services |
+-----------------+----+---------+----------------------------------+----------+
| <app-name>      | 1  | RUNNING | <app-name>.cloudfoundry.com      |          |
+-----------------+----+---------+----------------------------------+----------+

Customizing Spring Security Login using ExtJS in Grails Apps

I’ve been wanting to learn Grails for sometime and finally had the time to try out this web application framework. Grails is a very impressive framework, it is really about productivity and simplicity. Some of the benefits are scaffolding, testing framework, easy to create plugin and seamless integration with Spring and Hibernate. As part of my first attempt, I am customizing the Spring Security Login using ExtJS in a simple Grails application. The focus here is on the integration with Spring Security and ExtJS rather than Grails. I am using SpringSource Tool Suite (STS 2.7.1) to develop the demo application. The application is to display Quotes randomly and ability to maintain them. Let’s begin the integration and customization steps.

Step 1: Install the Spring Security Plugin

Spring Security is not built-in Grails, to begin the integration execute the following Grails command.

install-plugin spring-security-core

Step 2: Generate Domain Classes

 This step will create basic domain classes needed to store user information and the controller that handle the authentication. After executing the command below you will see several files being added to your project, such as LoginController, auth.gsp, denied.gsp, User, Role, UserRole, RequestMap. The domain class name for user and role is up to you to define and if you have an existing user domain, you must decide to integrate it (possibly by extending your domain class to the generated user class). Just a note that the RequestMap is an option and will look into it at later step.

s2-quickstart org.xaab.qotd User Role RequestMap

Step 3: Define URL mapping for login and logout

In order the make the login and logout controller reachable, the URL mappings must be added into UrlMapping.groovy

class UrlMappings {
	static mappings = {
		...
		"/login/$action?"(controller: "login")
		"/logout/$action?"(controller: "logout")
	}
}

Step 4: Add Access Control to the Controller

The need for this step depends on how you wants to control the access. You can use @Secured annotation to limit access to certain part of the application. In this example, I am not using annotation to limit the access, instead I define dynamics request maps which is shown in the Step 5. The following code is just an example of how @Secured can be used in the controller.

package org.example
import grails.plugins.springsecurity.Secured

class PostController {
    ...
    @Secured(['ROLE_USER'])
    def followAjax = { ... }

    @Secured(['ROLE_USER', 'IS_AUTHENTICATED_FULLY'])
    def addPostAjax = { ... }

    def global = { ... }

    @Secured(['ROLE_USER'])
    def timeline = { ... }

    @Secured(['IS_AUTHENTICATED_REMEMBERED'])
    def personal = { ... }
}

Step 5: Add Dynamic Request Maps

The request maps class was generated in Step 2 and used to limit access by defining the access control at the URL. Here I create the request map in BootStrap.groovy for demo purpose, in your actual application this may be defined in configuration file or database. I don’t think you should choose between using annotation and request map. You can strike balance by implementing both to handle general and fine-grain access control.

import org.xaab.qotd.*

class BootStrap {
    def springSecurityService

    def init = { servletContext ->
		...
		new Requestmap(url: '/**', configAttribute: 'IS_AUTHENTICATED_FULLY').save()
		new Requestmap(url: '/quote/**', configAttribute: 'ROLE_ADMIN,IS_AUTHENTICATED_FULLY').save()

		new Requestmap(url: '/js/**', configAttribute: 'IS_AUTHENTICATED_ANONYMOUSLY').save()
		new Requestmap(url: '/css/**', configAttribute: 'IS_AUTHENTICATED_ANONYMOUSLY').save()
		new Requestmap(url: '/images/**', configAttribute: 'IS_AUTHENTICATED_ANONYMOUSLY').save()
		new Requestmap(url: '/ext-4.0.2/**', configAttribute: 'IS_AUTHENTICATED_ANONYMOUSLY').save()
		new Requestmap(url: '/quote/random', configAttribute: 'IS_AUTHENTICATED_ANONYMOUSLY').save()
		new Requestmap(url: '/login/**', configAttribute: 'IS_AUTHENTICATED_ANONYMOUSLY').save()
		new Requestmap(url: '/logout/**', configAttribute: 'IS_AUTHENTICATED_ANONYMOUSLY').save()
		...
    }
    def destroy = {
    }
}

Step 6: Amend related GSP to add login and logout link

Amend related GSP (random.gsp) to ensure you have the login and logout link. Since I used scaffolding, I must install the templates and ensure the GSP templates are modified.  Once scaffolding templates are installed, you will see the folders /src/templates/scaffolding/**. The files I modified in this sample application are create.gsp, edit.gsp, list.gsp & show.gsp.

install-templates
...</pre>
<div class="nav"><span class="menuButton">Next Quote</span> <span class="menuButton">Admin</span> <span class="menuButton">Login</span> <span class="menuButton"> (Logout)</span></div>
<pre>...

Step 7: Customize the auth.gsp to Add ExtJs Login Form

By now the integration is completed and if you used the test data in Step 10, you will be able to see the default login page. In order to customize it, copy the ExtJS source into /web-app and in this sample application I am using ExtJS 4.0.2. Start the customization by modifying the auth.gsp as shown below. Do take note on the naming of the fields as it is the convention used in Spring Security.

Login<script type="text/javascript" src="${resource(dir:'ext-4.0.2',file:'ext-all.js')}"></script><script type="text/javascript">// <![CDATA[
		var defLoginUrl = '${postUrl}';
		var homeUrl = '${createLink(uri: "/quote/random")}';

// ]]></script>
<script type="text/javascript">// <![CDATA[
Ext.onReady(function(){
	Ext.QuickTips.init();

	var loginForm = Ext.create('Ext.form.Panel',{
			url: defLoginUrl,
			title: 'Login',
			renderTo: 'login',
			frame: true,
			cls: 'my-form-class',
			width: 350,
			items: [{
					xtype: 'textfield',
					fieldLabel: 'Login',
					name: 'j_username'
			},{
					xtype: 'textfield',
					inputType: 'password',
					fieldLabel: 'Password',
					name: 'j_password'
			}, {
				xtype: 'checkbox',
				fieldLabel: 'Remember Me?',
				name: '_spring_security_remember_me',
				checked: false
			}],
			buttons: [{
					id: 'lf.btn.login',
					text: 'Login',
					handler: function() {
						fnLoginForm(loginForm);
					}
				},{
					id: 'lf.btn.reset',
					text: 'Reset',
					handler: function() {
						fnResetForm(loginForm);
					}
			}]
	});

});

function fnLoginForm(theForm)
{
	theForm.getForm().submit({
		success: function(form, action) {
			Ext.Msg.alert('Success', 'Login Successful!', function(btn, text) {
				if (btn == 'ok') {
					window.location = homeUrl; //optionally this can be part of the data return by the server.
				}
			});
		},
		failure: function(form, action) {
			Ext.Msg.alert('Warning', action.result.error);
		}
	});
} //end fnLoginForm

function fnResetForm(theForm)
{
	theForm.getForm().reset();
} //end fnResetForm
// ]]></script>

Step 8: Configure Authentication Success and Failure Handler

The ExtJS login form will perform HTTP post to submit the login data however it expects JSON data as response. The default method in the LoginController will not be able to handle this, thus you must define your own handler. Add your handler into Config.groovy as shown below. The handlers are implemented in LoginController in the next step.

grails.plugins.springsecurity.successHandler.defaultTargetUrl = '/login/authSucccessExtJs'
grails.plugins.springsecurity.successHandler.alwaysUseDefault = true
grails.plugins.springsecurity.failureHandler.defaultFailureUrl = '/login/authFailExtJs?login_error=1'

Step 9: Implement the Handler in LoginController

After adding the handler, I suggest you remove or comment out the other methods to ensure they are not accessible.

import grails.converters.JSON

import javax.servlet.http.HttpServletResponse

import org.codehaus.groovy.grails.plugins.springsecurity.SpringSecurityUtils

import org.springframework.security.authentication.AccountExpiredException
import org.springframework.security.authentication.CredentialsExpiredException
import org.springframework.security.authentication.DisabledException
import org.springframework.security.authentication.LockedException
import org.springframework.security.core.context.SecurityContextHolder as SCH
import org.springframework.security.web.WebAttributes
import org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter

class LoginController {

	/**
	 * Dependency injection for the authenticationTrustResolver.
	 */
	def authenticationTrustResolver

	/**
	 * Dependency injection for the springSecurityService.
	 */
	def springSecurityService
	...
		/**
	 * The ExtJS Authentication success handler
	 */
	def authSucccessExtJs = {
		render([success: true, username: springSecurityService.authentication.name] as JSON)
	}

	/**
	 * The ExtJS Authentication failure handler
	 */
	def authFailExtJs = {
			def username = session[UsernamePasswordAuthenticationFilter.SPRING_SECURITY_LAST_USERNAME_KEY]
			String msg = ''
			def exception = session[WebAttributes.AUTHENTICATION_EXCEPTION]
			if (exception) {
				if (exception instanceof AccountExpiredException) {
					msg = SpringSecurityUtils.securityConfig.errors.login.expired
				}
				else if (exception instanceof CredentialsExpiredException) {
					msg = SpringSecurityUtils.securityConfig.errors.login.passwordExpired
				}
				else if (exception instanceof DisabledException) {
					msg = SpringSecurityUtils.securityConfig.errors.login.disabled
				}
				else if (exception instanceof LockedException) {
					msg = SpringSecurityUtils.securityConfig.errors.login.locked
				}
				else {
					msg = SpringSecurityUtils.securityConfig.errors.login.fail
				}
			}

			render([success: false, error: msg] as JSON)
	}
}

Step 10: Setup Test Data

So, you have integrated your Grails application, customized the login form using ExtJS. Now, setup the test data in BootStrap.groovy and launch the application (http://localhost:8080/grails-qotd-extjs).

import org.xaab.qotd.*

class BootStrap {
    def springSecurityService

    def init = { servletContext ->
		new Quote(author: "Unknown", content: "If you think sunshine brings you happiness, then you haven’t danced in the rain.").save()
		new Quote(author: "Lao Tzu", content: "An ant on the move does more than a dozing ox.").save()
		new Quote(author: "Tony Robbins", content: "It not knowing what to do, it’s doing what you know.").save()
		new Quote(author: "Bruce Lee", content: "Simplicity is the key to brilliance.").save()

		...

		def userRole = Role.findByAuthority('ROLE_USER') ?: new Role(authority: 'ROLE_USER').save(failOnError: true)
		def adminRole = Role.findByAuthority('ROLE_ADMIN') ?: new Role(authority: 'ROLE_ADMIN').save(failOnError: true)

		def adminUser = User.findByUsername('admin') ?: new User(
			username: 'admin',
			password: springSecurityService.encodePassword('admin'),
			enabled: true).save(failOnError: true)

		if (!adminUser.authorities.contains(adminRole)) {
			UserRole.create(adminUser, adminRole)
		}
    }
    def destroy = {
    }
}

Hope this tutorial helps and happy coding!

download source

References:

Display JSON Data using ExtJS 4 Store

I am sharing a short example of how to display JSON data using ExtJS 4 Store. In previous version of ExtJS, there is a small helper class call JsonStore which is used to simplify creation Stores from JSON data. In ExtJS 4 the class is no longer there. In this example, I am retrieving USGS data as JSON and display it in Grid as shown below.

USGS Data in Gri

As shown below in line 20, I use proxy of type JSONP in Ext.data.Store to retrieve the JSON data.

Ext.onReady(function(){
  Ext.define('UsgsList', {
    extend: 'Ext.data.Model',
    fields: [
       {name: 'fid',       type: 'int'},
       {name: 'title',     type: 'string'},
       {name: 'description',  type: 'string'},
       {name: 'link',      type: 'string'},
       {name: 'pubDate',    type: 'date'},
       {name: 'lat',      type: 'string'},
       {name: 'long',      type: 'string'}
    ],
    idProperty: 'fid'
});

var store = Ext.create('Ext.data.Store', {
    id: 'store',
    model: 'UsgsList',
    proxy: {
       type: 'jsonp',
       url: 'http://query.yahooapis.com/v1/public/yql',
    extraParams: {
       q: 'select * from rss where url="http://earthquake.usgs.gov/earthquakes/catalogs/eqs7day-M2.5.xml"',
       format: 'json'
   },
   reader: {
      root: 'query.results.item',
   }
 }
});

function renderTitle(value, p, record) {
   return Ext.String.format('<a href="{1}" target="_blank">{0}</a>',
   value,
   record.data.link
   );
}

var grid = Ext.create('Ext.grid.Panel', {
   width: 700,
   height: 500,
   title: 'USGS - M2.5+',
   store: store,
   loadMask: true,
   disableSelection: true,
   invalidateScrollerOnRefresh: false,
   viewConfig: {
     trackOver: false
   },
   // grid columns
   columns:[{
      xtype: 'rownumberer',
      width: 50,
      sortable: false
   },{
      id: 'title',
      text: "Title",
      dataIndex: 'title',
      flex: 1,
      renderer: renderTitle,
      sortable: false
   },{
      id: 'pubDate',
      text: "Published Date",
      dataIndex: 'pubDate',
      width: 130,
      renderer: Ext.util.Format.dateRenderer('n/j/Y g:i A'),
      sortable: true
   }],
   renderTo: Ext.getBody()
});

// trigger the data store load
store.load();
});

HTML5 @Google I/O 2011

Just sharing some of the talk/presentations on HTML5 at Google I/O 2011. HTML5 seems to be moving fast and you will see some of the cool stuff in the “HTML5 versus Android” demo,

  • I/O BootCamp 2011: Getting Started with HTML5

 

  • HTML5 Showcase for Web Developers: The Wow and the How

 

  • HTML5 and What’s Next

 

  • GWT + HTML5: A web developers dream!

 

  • HTML5 vs. Android: Apps or Web Mobile Development?

 

  • HTML5 Today with Google Chrome Frame

 

  • Kick-Ass Game Programming with Google Web Toolkit

 

 

Integrating JSR303 with Spring MVC 3 and ExtJS Forms

Previously in Spring MVC and ExtJS Forms I created a simple data entry form and there was no data validation implemented. Typically data validation occurs throughout application layers, from presentation to persistence layer. Here I will be using JSR-303 Bean Validation, where the data validation is defined in the domain model and can be used thoughout application layers. I will walk through how to integrate data validation into Spring MVC and will be using Hibernate Validator 4.x – which is the reference implementation for JSR-303 – to integrate into my previous example.

What is JSR-303?

“This JSR defines a metadata model and API for JavaBean validation. The default metadata source is annotations,
with the ability to override and extend the meta-data through the use of XML validation descriptors.
The validation API developed by this JSR is not intended for use in any one tier or programming model. It is specifically
not tied to either the web tier or the persistence tier, and is available for both server-side application programming,
as well as rich client Swing application developers.” – JSR-303 Specification

The data validation is achieved by defining constraints in the beans. Constraints are defined by a combination of constraint annotation and a list of constraint validation implementations. The constraint annotation can be applied at field-level, property-level (getter), class-level. Currently there are 22 built-in constraints for example @NotNull to check that the annotated value is not null or @Size(min=, max=) to check whether the annotated value lies between the specified range. You can also create your own custom constraint to meet your specific requirement. The concept here is similar to database constraints where the constraint can be applied to table column.

For now, I will walk through how to use the built-in constraints only. As usual, I will start with the configuration required to make the project work. Open the Maven pom.xml and add the following dependencies and repository:

<!-- JSR 303 with Hibernate Validator -->
<dependency>
	<groupId>javax.validation</groupId>
	<artifactId>validation-api</artifactId>
	<version>1.0.0.GA</version>
</dependency>
<dependency>
	<groupId>org.hibernate</groupId>
	<artifactId>hibernate-validator</artifactId>
	<version>4.1.0.Final</version>
</dependency>
...
<!-- For Hibernate Validator Repository -->
<repository>
	<id>org.jboss.repository.release</id>
	<name>JBoss Maven Release Repository</name>
	<url>https://repository.jboss.org/nexus/content/repositories/releases</url>
	<snapshots><enabled>false</enabled></snapshots>
</repository>

In the example, the form is submitted to HomeController and the form is mapped to PersonalContact bean. I added 3 built-in constraints into the PersonalContact bean, @NotBlank, @Patter (using regular expression) and @Email. When message is not define in the constraint annotation, the validator will assign a default value. The second part of the code below shows the validation process. The HomeController is autowired to the Validator bean and the validation() is executed inside the add().

public class PersonalContact implements Serializable
{
	private static final long serialVersionUID = 1L;
	private Long id;

	@NotBlank(message="Please enter name")
	private String name;
               // using regular expression to define the constraint
	@Pattern(regexp="[0-9]{8}", message="Please enter 8 digit phone number")
	private String phone;

	@Email
	private String email;

	public PersonalContact() {}

	public PersonalContact(Long id, String name, String phone, String email) {
		super();
		this.id = id;
		this.name = name;
		this.phone = phone;
		this.email = email;
	}
	//getter & setter
}
@Controller
public class HomeController
{
	@Autowired
	private Validator validator;

	@RequestMapping(value="/", method=RequestMethod.GET)
	public String home() {
		return "home";
	}

	@RequestMapping(value="/load", method=RequestMethod.POST)
	public @ResponseBody Map<String, ? extends Object> load(PersonalContact input) {
	...
	}

	@RequestMapping(value="/add", method=RequestMethod.POST)
	public @ResponseBody Map<String, ? extends Object> add(PersonalContact input, HttpSession session) {
		Map<String, Object> data = new HashMap<String, Object>();

		Set<ConstraintViolation<PersonalContact>> failures = validator.validate(input);
		if (!failures.isEmpty()) {
			//structure the response for ExtJS Form.
			data.put("success",Boolean.FALSE);
			data.put("errors", validationMessages(failures));
			data.put("errorMessage", "Add Failed!");
		} else {
			session.setAttribute(input.getName(), input);
			data.put("success",Boolean.TRUE);
		}

		return data;
	}
	//iterate to retrieve validation errors and store it in HashMap
	private Map<String, String> validationMessages(Set<ConstraintViolation<PersonalContact>> failures) {
		Map<String, String> failureMessages = new HashMap<String, String>();
		for (ConstraintViolation<PersonalContact> failure : failures) {
			failureMessages.put(failure.getPropertyPath().toString(), failure.getMessage());
		}
		return failureMessages;
	}
}

Since I used ExtJS forms, I must return JSON response in a specific format and the errors are mapped to the field and displayed as quicktips.


The spring-mvc-forms project is updated and available for download at myGit. Hope you find it useful. Happy coding! 🙂

References

{
   success: true,
   errors: { field1: 'error for field 1',
                 field2: 'error for field 2',
   	   ...
   }
}