Recently, I started working on a new Android project using Jetpack compose. I wanted to have two datetime field to capture start datetime and end datetime using the inbuilt pickers. The date and time selected from the picker should populate single text field with date and time together or two separate text fields representing the date and time respectively.

Much to my surprise there are no datetime picker composable as of now. Furthermore, I realized that  DatePickerDialog and DatePicker exists but managing populating the text field with DatePicker state needs to be done by custom code. Finally, for time, TimePicker exists but there is no out of the box TimePickerDialog as of now. In the post, I am sharing the custom datetime composable that I created to accomplish my requirements.

DateTimePicker Preview

The below code when added inside a column will render the two datetime fields.

       CustomDateTimePicker(
            title = "Start datetime",
            state = startDateTimeState
        )
        CustomDateTimePicker(
            title = "End datetime",
            state = endDateTimeState
        )

The complete running code can be downloaded from the github repository.

Github Repo: MyDateTimePicker App

High-level design

Below are the different components written to create and display the datetime composable.

DateTimePicker.kt

The method CustomDateTimePicker is the top-level composable that would generate a datetime composable. It internally calls the CustomDatePicker and CustomTimePicker composable. These two composables utilize the Dialog and Picker to display the date or time strings in a text field. The CustomDateTimePicker uses an mutable state object of type of LocalDateTime class. This object is set with the date and time picked by the user and displayed in the date and time text fields.

TFDateUtils.kt

The object provides different utility methods for manipulating LocalDate, LocalDateTime, String and Long milliseconds. These methods are used from the different composable methods in DateTimePicker.kt.

Code walk-through 

The DatePickerState and TimePickerState manage the state of the Date and Time picker. Initialized differently, I created mutable state strings to remember the date and time value to be displayed on the text fields. Ideally, I wanted to use the DatePickerState and TimePickerState to directly set the value in the text fields. But, these objects updated even when the user uses the picker, selects the data or time and cancels or dismisses the dialog. 

val displayDate = remember {
        mutableStateOf(TFDateUtils.getCurrentDateString(dateFormat))
    }

As mentioned earlier, jetpack compose does not have a TimePickerDialog. I used the composable written in the article Exploring Date and Time Pickers: Compose Bytes.

Lastly, in the CustomDateTimePicker composable, I created a lambda function to update the LocalDateTime with the DatePickerState and TimePickerState value (or the default initial values). This function is passed to the CustomDatePicker and CustomTimePicker composable, so the function can be called every time the user selects a new date or time.  

val updateState = {
        val dateString = if (dateState.selectedDateMillis != null) {
            TFDateUtils.convertMillisToDateString(
                millis = dateState.selectedDateMillis!!,
                format = dateFormat
            )
        } else {
            TFDateUtils.getCurrentDateString(dateFormat)
        }
        val timeString = if (timePickerState.minute != null 
                                               && timePickerState.hour != null) {
            TFDateUtils.getTimeString(
                hour = timePickerState.hour,
                min = timePickerState.minute
            )
        } else {
            TFDateUtils.getCurrentTimeString(timeFormat)
        }
        state.value = TFDateUtils.getLocalDataTimeFromString(
            dateTime = "$dateString $timeString",
            format = dateTimeFormat
        )
    }

I have used a Toast to display the value of the LocalDateTime state set by the CustomDateTimePicker composable. In the real-world, this will probably need a converter to convert to string and save it to Rooms database through a ViewModel. 

Known issues and improvements

• The current implementation has some glitch when either date or time is selected but not both. The LocalDateTime state object needs to be synchronized better with the displayDate and displayTime mutable state strings to prevent this issue.
• Using OffSetDateTime instead of LocalDateTime will help account for timezone, especially when the device moves between timezones. 

Hope the article provided a good starting point to create a custom datetime picker composable that can be reused by quickly adding to other composable.

Recently, I have been working on manipulating xmls and csvs using Python. I was amazed by how quick and easy it is manipulate these files in Python. In this article I cover the flexibility of the python and xml.etree.ElementTree API. Let us jump into it.

Problem

Given a dictionary of words and meaning in multiple language. We want to manipulate the xml and add another meaning in French language for the word beauty. 

<dictionary>
    <definition>
        <word lang="en-US">beauty</word>
        <meaning lang="en-US">
            a combination of qualities, such as shape, color, or form, that pleases the aesthetic senses, especially the sight.
        </meaning>
    </definition>
</dictionary>

Approach 1

The simplest approach is to find / reach the element of definition. Then add a sub-element meaning to the definition. This would look something like below.

import xml.etree.ElementTree as ET

# Parse the XML file
tree = ET.parse('sample.xml')
root = tree.getroot()

beauty_meaning_fr = "La beauté est l'harmonie des formes et des couleurs qui évoque un sentiment d'émerveillement et d'admiration."

def_elements = root.findall('definition')

for definition in def_elements:
    word = definition.find('word')
    if word.text == 'beauty':
        new_meaning = ET.SubElement(definition, 'meaning')
        new_meaning.text = beauty_meaning_fr
        new_meaning.set('lang', 'fr-FR')

tree.write('output.xml')

This approach works great and provides the desired output.

<dictionary>
    <definition>
        <word lang="en-US">beauty</word>
        <meaning lang="en-US">
            a combination of qualities, such as shape, color, or form, that pleases the aesthetic senses, especially the sight.
        </meaning>
    <meaning lang="fr-FR">La beaut&#233; est l'harmonie des formes et des couleurs qui &#233;voque un sentiment d'&#233;merveillement et d'admiration.</meaning></definition>
</dictionary>

However, when I was working on my real-life problem, the above code did not add the meaning element for some reason. Hence, I started looking for another approach to solve my issue.

Approach 2

Another option is to add a new element meaning, instead of a sub-element. Of course, the element will then need to added to another element. To achieve our requirement, we need to reach the definition element and insert or append the meaning element that we initially created. The below code summarizes this approach:

import xml.etree.ElementTree as ET

# Parse the XML file
tree = ET.parse('sample.xml')
root = tree.getroot()

beauty_meaning_fr = "La beauté est l'harmonie des formes et des couleurs qui évoque un sentiment d'émerveillement et d'admiration."

def_elements = root.findall('definition')

for definition in def_elements:
    word = definition.find('word')
    if word.text == 'beauty':
        new_meaning = ET.Element("meaning")
        new_meaning.text = beauty_meaning_fr
        new_meaning.set('lang', 'fr-FR')
        definition.append(new_meaning)

tree.write('output1.xml')

The code works as expected and generates the output similar to that of approach 1.

Requirement

Create a RESTful service using Spring MVC and Hibernate for fetching, adding, updating and deleting product information.  The product information will be persisted in a database table. The service should be available to be consumed by any client like mobile, html, java application etc. using the REST url created.

GitHub Project URL

https://github.com/tech-freaks/catalog-service

Pre-requisites

• MySQL and Tomcat is already installed
• Eclipse with JEE support is already installed
• Basics about Spring and Hibernate framework
• Basics about RESTful service

Analysis and Design

We will use a bottom-up approach to design our application. We start from the database layer, then hibernate layer and finally the Spring layer.

Database

For implementing this requirement, we create a table PRODUCT. It has basic information about a product like name, description, partnumber, unit price etc. We do not have any more table for fulfilling this requirement. However, if you create a complex application, your product information might be normalized into multiple related tables.

Below is the CREATE table statement for creating the PRODUCT table.

 CREATE TABLE IF NOT EXISTS `product` (
  `product_id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(50) NOT NULL,
  `partnumber` varchar(50) NOT NULL,
  `description` varchar(1024) NOT NULL,
  `thumbnail_url` varchar(240) DEFAULT NULL,
  `buyable` char(1) NOT NULL DEFAULT 'Y',
  `unitprice` decimal(8,2) NOT NULL,
  PRIMARY KEY (`product_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

There is two sql script in the github repository root folder create_tables.sql and product_load.sql. As the name suggests, the first sql can be used to create a database and PRODUCT table. The second sql can be run to insert some starter products.

Hibernate

Nothing to think about here! We will have a hibernate class Product.java which will be mapped to PRODUCT table. If you would want to reverse engineer the Product.java from the database table, review the steps in the article Reverse Engineering Hibernate objects using JBoss eclipse plugin

Spring

We will have a single Controller, which will have different methods for each verb like GET, PUT, POST and DELETE and will act upon Product table data. Following the Spring convention, we will interact with the database using the following layered approach

Spring Controller -> Service -> DAO -> Hibernate -> Database

Spring Controller:
CatalogRestController

Service:
ProductService
ProductServiceImpl

DAO:
ProductDAO
ProductDAOImpl

Furthermore, the Rest controller will have the following methods

@RequestMapping(method = RequestMethod.GET)
public List<Product> getProducts(); - Get all products

@RequestMapping(value = "/{productId}", method = RequestMethod.GET)
public ResponseEntity<?> getProduct(@PathVariable Integer productId) <- Get Products with ProductId

@RequestMapping(method = RequestMethod.POST)
public ResponseEntity<?> add(@RequestBody Product input) <- Insert a product for input received

@RequestMapping(value = "/{productId}", method = RequestMethod.PUT)
public ResponseEntity<?> update(@PathVariable Integer productId,
@RequestBody Product input) <- Update product information for productId

@RequestMapping(value = "/{productId}", method = RequestMethod.DELETE)
public ResponseEntity<?> remove(@PathVariable Integer productId) <- Delete product for the provided productId

The above also shows at a high level usage of Spring annotations for specifying different verbs.

Word of caution: This design allows a REST client to insert, delete or update records in database, without any authentication and authorization. For a production application, generally, the only verb which might allow access without authorization is the GET. All the other verbs which modify the database might be authenticated and authorized. Oauth is a commonly used open standard of authorization used to authorize client before they can make changes to the underlying data source.

We will need to further create XML configuration file for web application, Spring framework and add libraries. We will cover this in environment setup and development sections.

Environment Setup

• Open eclipse and create a “Dynamic Web Project” in eclipse with the name “catalog-service”.
• Click next, then next and finish.
• If you are using Maven or Gradle, setup to download libraries for Hibernate 4 or above, Spring 4.1 or above and MySQL drivers. If you are using Spring 4.1, make sure to also include the latest libraries for Jackson FastXML. The conversion of Object to JSON and vice versa is handle by Jackson library and it does not work without the latest libraries.
• If you are not using maven or gradle, make sure the following jars are in your WEB-INF/lib folder as provided in the below screenshot

The source code in the GitHub contains all required libraries.

Web.xml configuration

We configure Spring DispatcherServlet, specify the patch to Spring configuration file (if not using default path and file name) and map the request to a URL pattern. For this project, we use *.json mapping. Hence, all the request URL should have .json at the end. Note that, if you perform a partial URL mapping like /product/*, you will have to configuring the ViewResolver, so that it could use the appropriate resolvers for JSON. However, when an extension mapping like .json, no ViewResolver configuration is required.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xmlns="http://java.sun.com/xml/ns/javaee" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" id="WebApp_ID" version="3.0">
  <display-name>Catalog REST Services</display-name>
  <welcome-file-list>
    <welcome-file>Login.jsp</welcome-file>
  </welcome-file-list>
  <servlet>
    <servlet-name>CatalogServlet</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <init-param>
      <param-name>contextConfigLocation</param-name>
      <param-value>/WEB-INF/Catalog-Servlet.xml</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>CatalogServlet</servlet-name>
    <url-pattern>*.json</url-pattern>
  </servlet-mapping>
</web-app>

WEB-INF/Catalog-Servlet.xml

Most of the tags are self-explanatory. Although this is a Spring configuration, it references Hibernate related files and hibernate.cfg.xml. The “datasource” needs to be updated with your username, password and dbname. Also, note the configuration for transactionManager. Once this is configured, Spring handles the hibernate transactions and you do not have to manually manage the transaction in your code. Just mark the service method with @Transactional annotation.

The ProductDao and ProductService is also declared. We have marked them as @Autowired and hence we do not have to pass the reference for productDao into productService in the XML.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:context="http://www.springframework.org/schema/context"
	xmlns:mvc="http://www.springframework.org/schema/mvc"
	xmlns:tx="http://www.springframework.org/schema/tx"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.1.xsd
		http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.1.xsd
		http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-4.1.xsd
		http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.1.xsd">

	<mvc:annotation-driven />
<!-- Configure JDBC Connection - @TODO update url, username and password per your database configuration -->
      <bean id="dataSource"
            class="org.springframework.jdbc.datasource.DriverManagerDataSource">
            <property name="driverClassName" value="com.mysql.jdbc.Driver" />
            <property name="url" value="jdbc:mysql://localhost:3306/tfcom_articles" />
            <property name="username" value="admin" />
            <property name="password" value="admin*" />
      </bean>
      
    <tx:annotation-driven transaction-manager="transactionManager" />
    
     <bean id="transactionManager"
            class="org.springframework.orm.hibernate4.HibernateTransactionManager">
        <property name="sessionFactory" ref="sessionFactory"/>
    </bean>
    
      
    <bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
    <property name="dataSource">
                  <ref bean="dataSource" />
            </property>
    <property name="configLocation" value="classpath:hibernate.cfg.xml" />
	</bean>
	
	<context:component-scan base-package="com.tech_freaks.catalog"/>
	
	<bean id="productDao"
   	class="com.tech_freaks.catalog.dao.ProductDAOImpl"/>
   	
   	<bean id="productService"
   	class="com.tech_freaks.catalog.service.ProductServiceImpl"/>
</beans>

src/hibernate.cfg.xml

We have used the XML configuration for hibernate and not annotations. This mainly declares the Product hibernate class and its mapping file Product.hbm.xml.

The commented fields in this file will need to be un-commented, if you are reverse engineering the hibernate object as explained in the article Reverse Engineering Hibernate objects using JBoss eclipse plugin

Development

Hibernate Object
Product.java – We reverse engineer this class from the database. It is the model for the application, which is mapped to the database table. It is self-explanatory.

DAO layer

AbstractDAO<T> – This is a generic class which contains common calls to hibernate API. All DAO can extend this class to reuse these pre-implemented class. Note, how the sessionFactory is injected by Spring framework into the DAO layer using @Autowired annotation. The sessionFactory is declared in the Catalog-Servlet.xml file which we defined earlier.

The JavaDoc comments on each of the method, explain the use of the function.

package com.tech_freaks.catalog.dao;

import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.springframework.beans.factory.annotation.Autowired;

/**
 * Generic Base class which encapsulates hibernate specific calls which will be available from the 
 * extending DAO which is mapped to the database table.
 * @author Tech Freaks
 *
 * @param <T> - The Hibernate object which it works on
 */
public class AbstractDAO <T> {
	
	@Autowired
    private SessionFactory sessionFactory;
 
	/**
	 * Gets a current session from the hibernate factory
	 * @return
	 */
    protected Session getSession(){
        return sessionFactory.getCurrentSession();
    }
    
    /**
     * Generic method for finding by primary key. 
     * @param id  - Primary key value in integer
     * @param myClass - Type of Class which will be loaded
     * @return T representing instance of the loaded hibernate class instance
     */
    public <T>  T findById(Integer id, Class myClass) {
    	return (T) getSession().load(myClass, id);
    }
 
    /**
     * Inserts record into database
     * @param t - Instance of the Hibernate object which needs to be persisted in the db
     * @return - Key value of the newly inserted record
     */
    public Integer persist(T t) {
       return (Integer) getSession().save(t);
    }
    
    /**
     * Updates the database for provided generic type
     * @param t Generic type for the hibernate object
     */
    public void update(T t) {
    	getSession().update(t);
    	getSession().flush();
    }
     
    /**
     * Deletes the entity from the database for the provided generic type
     * @param t  Generic type for the hibernate object
     */
    public void delete(T t) {
        getSession().delete(t);
        getSession().flush();
    }
}

ProductDAOImpl & ProductDAO
This is the actual DAO interface and its implementation. It extends the AbstractDAO<Product> and utilizes the methods defined there.

package com.tech_freaks.catalog.dao;

import java.util.List;

import org.hibernate.Session;
import org.hibernate.SessionFactory;

import com.tech_freaks.catalog.model.Product;

public class ProductDAOImpl extends AbstractDAO<Product> implements ProductDAO {
	
	
	/**
	 * Fetches all the product from the PRODUCT table
	 */
	@Override
	public List<Product> getAllProducts() {
		Session session = getSession();
        List<Product> productList = session.createQuery("from Product").list();
        return productList;
	}
	
	/**
	 * Fetches product based on the primary key productId
	 */
	@Override
	public Product findById(Integer id) {
		return (Product) getSession().get(Product.class, id);
	}
	
	/**
	 * Inserts a new record in the product table.
	 */
	@Override
	public Integer insert(Product product) {
		return persist(product);
	}
	
	/**
	 * Loads the product using the primary key and then removes it from the database.
	 */
	@Override
	public boolean remove(Integer id)  {
		boolean deleted = false;
		Product delProduct = findById(id);
		if(delProduct!=null) {
			delete(delProduct);
			deleted = true;
		} 
		return deleted;
	}
}

ProductServiceImpl & ProductService

The is the Service layer interface and its implementation for the Product manipulation. It internally calls the DAO layer. The Service layer is called from the Controller class. The methods are self-explanatory.

The below excerpt from ProductServiceImpl shows @Autowiring annotation on productDAO, which is injected in by the Spring framework. Also, each method is marked with annotation @Transactional. This is the reason, you do not see any code related to transaction management in the DAO layer. Spring manages the transaction code for you.

...
@Autowired
	private ProductDAO productDAO;
	
	@Override
	@Transactional
	public List<Product> listProducts() {
		
		return productDAO.getAllProducts();
	}
...

CatalogRestController is the entry point into the code for all REST service verbs. The @RestController annotation eliminates the need to mark each method with @ResponseBody when it returns an Object or a Collection of objects. Now even without @ResponseBody, when an object or collection of object is returned, it is automatically converted into JSON. The class is also marked with @RequestMapping(“/products”). This means the REST URL for every verb call will have ‘/products’ in it, after the application context name. Now, each method is marked with @RequestMethod. We follow the REST specification by mapping GET to retrieve data, POST for insert data, PUT for update data and DELETE for deleting data.

If we look at the method “add”, there are certain points worth noticing.

First, after every REST call, we return a HttpStatusCode. If the insert is successful, we return CREATED status or 201. If it fails, because the Product object was not populated, it means, the JSON was not well formed. In this case, we return BAD_REQUEST or 400. This helps the client to read the HttpHeader in the response and take appropriate action.

Second, after insert, the method create a REST URL to fetching the newly inserted record. This is set in the Location HttpHeader.

The private method isValid() is called internally to perform some basic validation before the service layer is invoked.

Similarly, all the other method are developed. The complete CatalogRestController code is below.

package com.tech_freaks.catalog.controller;

import java.net.URI;
import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.hibernate.HibernateException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.util.StringUtils;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.support.ServletUriComponentsBuilder;

import com.tech_freaks.catalog.model.Product;
import com.tech_freaks.catalog.service.ProductService;

@RestController
@RequestMapping("/products")
/**
 * Entry point for REST service calls. Different method for handling insert, update, delete and get
 */
public class CatalogRestController {

	protected final Log logger = LogFactory.getLog(getClass());

	@Autowired
	private ProductService productService;

	@RequestMapping(method = RequestMethod.GET)
	public List<Product> getProducts() {
		return productService.listProducts();

	}

	@RequestMapping(value = "/{productId}", method = RequestMethod.DELETE)
	public ResponseEntity<?> remove(@PathVariable Integer productId) {
		boolean deleted = false;
		ResponseEntity<?> response = null;
		try {
			deleted = productService.removeProduct(productId);
		} catch (Exception e) {
			logger.error("Error deleting product from database: "
					+ e.getMessage());
		}
		if (deleted)
			response = new ResponseEntity<>(HttpStatus.OK);
		else
			response = new ResponseEntity<>(HttpStatus.NO_CONTENT);
		return response;
	}

	@RequestMapping(value = "/{productId}", method = RequestMethod.GET)
	public ResponseEntity<?> getProduct(@PathVariable Integer productId) {
		ResponseEntity<Product> response = null;
		Product product = productService.getProduct(productId);
		if (product != null) {
			response = new ResponseEntity<>(product, HttpStatus.OK);
		} else {
			response = new ResponseEntity<>(HttpStatus.NO_CONTENT);
		}
		return response;
	}

	@RequestMapping(value = "/{productId}", method = RequestMethod.PUT)
	public ResponseEntity<?> update(@PathVariable Integer productId,
			@RequestBody Product input) {
		ResponseEntity<?> response = new ResponseEntity<>(HttpStatus.OK);
		logger.info("Product Id to be updated is " + productId);
		logger.info("New Product Name will be " + input.getName());
		try {
			if (productId > 0 && isValid(input)) {
				input.setProductId(productId);
				productService.updateProduct(input);
				response = new ResponseEntity<>(HttpStatus.OK);
				
			} else {
				response = new ResponseEntity<>(HttpStatus.BAD_REQUEST);
			}
		} catch (HibernateException he) {
			logger.error("Unable to update product in database: "+ he.getMessage());
			response = new ResponseEntity<>(HttpStatus.NO_CONTENT);
		} catch (Exception e) {
			logger.error("Unable to update product in database: "+ e.getMessage());
			response = new ResponseEntity<>(HttpStatus.BAD_REQUEST);
		}
		return response;
	}

	@RequestMapping(method = RequestMethod.POST)
	public ResponseEntity<?> add(@RequestBody Product input) {
		HttpHeaders httpHeaders = new HttpHeaders();
		ResponseEntity<?> response = null;
		if (isValid(input)) {
			logger.info("Product description is " + input.getDescription());
			try {
				Integer productId = productService.createProduct(input);
				URI location = ServletUriComponentsBuilder
						.fromCurrentServletMapping()
						.path("/products/{productId}.json").build()
						.expand(productId).toUri();
				logger.info("Location uri is " + location.toString());
				httpHeaders.setLocation(location);
				response = new ResponseEntity<>(null, httpHeaders,
						HttpStatus.CREATED);
			} catch (Exception e) {
				logger.error("Unable to insert new product in database: "
						+ e.getMessage());

				response = new ResponseEntity<>(null, httpHeaders,
						HttpStatus.UNPROCESSABLE_ENTITY);
			}
		} else {
			response = new ResponseEntity<>(null, httpHeaders,
					HttpStatus.BAD_REQUEST);
		}
		return response;
	}

	/*
	 * Internally used to validate the fields to make sure, all required fields
	 * are present before calling the service layer for insert
	 */
	private boolean isValid(Product product) {
		boolean valid = false;
		if (product != null) {
			if (!StringUtils.isEmpty(product.getName())
					&& !StringUtils.isEmpty(product.getDescription())
					&& !StringUtils.isEmpty(product.getPartnumber())
					&& !StringUtils.isEmpty(product.getName()))
				valid = true;
		}
		return valid;
	}
}

At this point, you can download the complete project from GitHub at https://github.com/tech-freaks/catalog-service

Unit Testing

The first part of the unit test, is to figure out the REST URL to invoke. We can figure this out, if we look at the CatalogRestController and web.xml. Remember, web.xml maps *.json to the controller. Hence, for all verbs our URL will end with .json.

We used the following information:

Tomcat host: http://localhost:8080
Application Context: /catalog-service
CatalogRestController mapping: /products

Now, the method getProducts() do not have any value mapping. But, it needs to be a GET call. Hence, for accessing this method, you will use the URL http://localhost:8080/catalog-service/products.json and it needs to be a GET call. You can therefore, just paste the URL in a browser and it will show the JSON with all products.

Similarly, if you look at getProduct(), the request mapping has /{productId}. This is equivalent to the primary key, product_id value in the database. Hence, the URL will be http://localhost:8080/catalog-service/products/1.json , where product_id=1. This will be again a GET verb again, as we are fetching information.

Now, similarly, we can figure out the remaining URL for calling remove(), update() and add(). The below table provide the remaining URL and verbs.

Where {productId} will be replaced by product_id value in the database.

Now, there will be two question:
1. How to invoke DELETE, PUT and POST verb? When we paste the URL in the browser, they all are by default GET requests.
2. How to pass the product information for add() and update()?

For # 1, we can download a browser plugin called as Poster (Just search Poster plugin), which is available for Chrome and Firefox. The below screenshot shows, Poster in action for Firefox.

For # 2, we need to send an input json containing the product information to be insert or updated in the product table. The POST performs and insert, while PUT updates. The json format is exactly as the response received from http://localhost:8080/catalog-service/products/{productId}.json GET verb. However, we have provided an input file by the name POST_PUT_Input.json, which is in the root folder of the project.

In Poster plugin, you need to make sure, that to add a Header by the name “Content-Type” with value “application/json”.

For integration testing, we generally run the client code like the mobile application in same network and point to the URLs provided above. However, integration testing is beyond the scope of this article.

This completes the workshop for creating a simple product REST service using Spring MVC 4.

Happy RESTing!

Overview

When you are working on a web application development using Tomcat server and Eclipse IDE, it could be painful and inefficient to create a WAR file and deploy to Tomcat after every code change. There are multiple ways to make the development process efficient.

1. Create Tomcat Server within Eclipse
2. Create Ant or other build script to deploy to Tomcat webapps folder
3. Setup your Eclipse project to be within Tomcat/webapps folder

The simplest and most convenient is the creating a Tomcat Server within Eclipse. This article walks through the steps to configure Tomcat to run from within Eclipse.

This article uses Eclipse Kepler and Tomcat 8, but the process should be similar with other versions too.

The article does not walk through JDK, Tomcat and Eclipse installation.

Steps

1. Download and install Java JDK 7 or later from Oracle website: http://www.oracle.com/technetwork/java/javase/downloads/index.html
2. Download and install Tomcat 8 from Apache website: https://tomcat.apache.org/download-80.cgi

4. Download Eclipse for JEE developer from Eclipse website: http://www.eclipse.org/downloads/packages/eclipse-ide-java-ee-developers/keplersr1
5. Follow the URL https://wiki.eclipse.org/JDT/Eclipse_Java_8_Support_For_Kepler if you installed JDK 8 and want Eclipse to support JDK 8
6. Creating Tomcat 8 server in Eclipse

a. Open Java EE perspective and access the Server tab below. You should see as the below screenshot

b. Click the link to create new Server as highlighted in screen above.

c. In the window that opens, click Apache and check for Tomcat 8. If you do not see, Tomcat 8 option perform the steps d to e.

d. Go to Help -> Install New Software..

e. In the window that opens, enter the URL as – http://download.eclipse.org/webtools/repository/luna

 >> Under the Web Tools Platform (WTP) 3.6.0, select options “JST Server Adapters” and “JST Server Adapters Extensions”. Click Next and then Accept the license agreement and finish installation.

 >> Eclipse will prompt to restart and after restarting, you should be able to Tomcat 8 option

f. Now select Tomcat 8 and click next

g. In the next screen, select the location of Tomcat Installation and also JDK, as shown below

h. Click next and you should see the option to add your web application. Add a test application, if you have one and click finish.

i. You should see the new Tomcat 8 server and displayed in the Server window as shown below

j. Double click the new Server definition and in the file which opens in Eclipse make the settings as highlighted in the below screen. The selected option uses the Tomcat installation path to work with, instead of create a separate set of files in the user folder. If you do not want Eclipse to work on the installation folder of Tomcat, you might want to select the first option ‘Use Workspace Metadata’.

k. Next right click and Server and click Start. This should start your Tomcat server without any errors, in the Console.

l. Verify the Tomcat server by access the URL http://localhost:8080 and you should be able to see Tomcat screen.

This concludes the walk-through of configuring Tomcat to be access from within Eclipse IDE. Enjoy developing, deploying and testing on your development environment completely from within Eclipse !

If you notice error while starting Tomcat which says “Access Denied”, it means that Eclipse is not having permission to access file in the Tomcat installation folder. You can resolve this by giving required permission to the Tomcat installation folder or by running your Eclipse as Administrator. You can also use the option ‘Use Workspace Metadata’, in step j, if you are unable to resolve the access issue.

Writing hibernate class and xmls can be a pain, especially, if you have tons of tables in your database. One way to alleviate the pain is to use a reverse engineering tool, which would generate the xml and java files for hibernate by connecting to database. This article walks through the steps from installing the JBoss Tools Eclipse plugin to generate hibernate java and xml files.

Pre-requisites 

• MySQL and Eclipse are installed
• MySQL has a database with at least one table
• You have some basic knowledge about hibernate or any other ORM tool

Steps

Installing JBoss Eclipse Tools from marketplace

Open Eclipse and follow the menu, Help -> Eclipse Marketplace..

Search for ‘JBoss’ or ‘Hibernate’ and click install on the Jboss Tools

Un-select all options except “Hibernate Tools” as shown in the below screenshot

Click “Confirm” and then “Finish”. Accept the license agreement.

Eclipse will prompt to restart

Creating the hibernate configuration (hibernate.cfg.xml)

Once Eclipse has restarted, Open hibernate perspective. Windows -> Open Perspective -> Others

The perspective will be empty. Click on the “Add Configuration” to start the configuration process, as shown below

A window “Edit Configuration” will show up, as per the below screen shot. Select the Project by clicking Browse.

Next, click “Setup” from Configuration File section. This will open another windows as shown below.

Using this, we will be creating the hibernate.cfg.xml which has the information like the database dialect, database URL, drivers, database type, db user id and db password. Once you click “Finish”, this will generate the hibernate.cfg.xml. Then click the “OK” in the “Edit Configuration” window should be open.

Next, go to the Hibernate Configuration in the Hibernate perspective. Right click the new configuration which you added now and click “Rebuild Configuration”. You should now check the section under “Database” in the hibernate configuration.  You should see the tables from your database. If the tables are not showing and it shows an error, you should not proceed further and resolve this issue first. Below are some possible reasons for the error

• Database drivers are not in classpath:  Resolution: Put the driver jar file in the WEB-INF/lib folder to resolve this.
• Database service is not started: Resolution: Start your database and try connecting again.
  One or more information entered in the hibernate.cfg.xml is incorrect. Verify all the information entered in the hibernate.cfg.xml, anything which is incorrect.
• If you resolved the issue as explained above, you might want to “Rebuild Configuration” after resolving the issue.Once you can see the Database tables, you can proceed to the reverse engineering steps provided below.

  Create a new Hibernate Reverse Engineering file

  From the Hibernate Perspective click “File” -> “New” -> “Hibernate Reverse Engineering File (reveng.xml)”

In the Wizard that opens, select the project and click “Next”

Select the “Console Configuration” from the drop down in the next step and click the “Refresh” button. The Database Schema section will show your database and tables. Select all the tables and click include. Click the “Finish” button to generate hibernate.reveng.xml file.

If you are unable to generate the hibernate objects, check that you have included all individual tables instead of the whole database and re-generate the reveng.xml

Once you have the reverse engineering file generated, you need to do Run -> Hibernate Code Generation -> Hibernate Code Generation Configuration€¦

In the window that opens, create a new configuration and enter details as per the below screenshot and click “Run”.

Go back to Java EE perspective and you will see the hibernate xmls and java files generated in the package specified in the Main tab of the configuration before clicking Run.

You might want to modify some of the content in the xml or java files generated as per your needs. However, remember that, if you have to regenerated the hibernate objects, your manual changes might get lost.

One last step you would need to perform, is to add the mapping of the bean in the hibernate.cfg.xml. Below shows a screenshot of adding one mapping.

That completes the steps to reverse engineer hibernate objects using JBoss Tools in Eclipse.

Requirement

• Create a Login page to enter user name and password
• On submit, validate the user name / password against MySQL database
• If the authentication is successful, forward to home page showing welcome message along with the user name
• If the authentication fails, return back to the login page with appropriate error message
• If there is exception / errors during authentication process return back to login page with appropriate error message.

Pre-requisites

• MySQL and Tomcat is already installed
• MySQL is setup with database and tables for use. Refer Simple Login Application for details
• You have basic knowledge of Java and J2EE web applications
• Basic idea about Struts 1 or 2 or at least about MVC design

Concepts Covered

• IDE set up for Struts 2
• Struts2 key components
• Difference between struts 1 and struts 2 highlighted

Environment Setup

• Download Eclipse for Java EE which is all ready to go with JEE features and install it. Download from Eclipse for JEE Download
• Create a “Dynamic Web Project” in eclipse with the name Struts2LoginApp.
  Click next, then next and finish.
• Download latest Struts2 binaries from Struts 2 Binaries Download. At the time of writing this article, struts 2.3.24 was used.
• Unzip the binaries and copy the following JAR files to the lib directory(<workspace>\Struts2LoginApp\WebContent\WEB-INF\lib) of the newly created project:

commons-fileupload-1.3.1.jar
commons-io-2.2.jar
commons-lang3-3.2.jar
commons-logging-1.1.3.jar
commons-logging-api-1.1.jar
freemarker-2.3.22.jar
javassist-3.11.0.GA.jar
ognl-3.0.6.jar
struts2-core-2.3.24.jar
xwork-core-2.3.24.jar

Since we make connections to MySQL using JDBC, we would also need to put the JAR for the MySQL JDBC driver (mysql-connector-java-5.1.20-bin.jar). You can download the latest version of the jar from MySQL JDBC Driver Download.

web.xml configuration:
A blank web.xml might have been added to your project under WEB-INF folder. This file is the JSP deployment descriptor. We need to configure the struts 2 controller, which is a servlet filter by the name FilterDispatcher. If you are familiar with Struts1, you will recollect that in the web.xml, we used to make an entry for the ActionServlet which was the controller in struts1.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="http://java.sun.com/xml/ns/javaee" 
   xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
   xsi:schemaLocation="http://java.sun.com/xml/ns/javaee 
   http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
   id="WebApp_ID" version="3.0">
   
   <display-name>Struts 2 Login App</display-name>
   <welcome-file-list>
      <welcome-file>Login.jsp</welcome-file>
   </welcome-file-list>
   <filter>
      <filter-name>struts2</filter-name>
      <filter-class>
         org.apache.struts2.dispatcher.FilterDispatcher
      </filter-class>
   </filter>

   <filter-mapping>
      <filter-name>struts2</filter-name>
      <url-pattern>/*</url-pattern>
   </filter-mapping>
</web-app>

Analysis and Design

If you do not have the database table created or do not have any user in that table, please refer the analysis and design from the article Simple Login Application for details.

Taking inspiration from the previous article, we will create similar components for our simple login app using struts 2.

So, below are the identified components:

Login.jsp
Home.jsp
LogonAction.java

Let us look at each identified component and its role in our application:

• Login.jsp -> It would present the login screen to enter user id and password. It also displays any validation errors and login failure message.
• Home.jsp -> Simple JSP which displays a welcome message after successful login
• com.tech_freaks.action.LogonAction.java
  While we take inspiration from LogonServlet of our previous example, we improve the design to not include any database connectivity code in our Action class. To achieve this, we will refactor all the DB access / calls to a Helper class by the name DBHelper.java. This also helps in modifying the code, if we tomorrow decide to use Entity Beans (EJB) or hibernate to access the database instead of raw JDBC code with minimum impact on our Action class. The LogonAction performs the following tasks.
  – Perform basic empty check on server side before calling the database
  – Call our DB Helper class, with user name and password to check, if the combination exists.
• com.tech_freaks.utils.DBHelper.java
  It will perform all task related to getting connection to the DB, executing a query and method to return Boolean indicating, if record was available or not in the DB. We do not write method like, isUserPasswordValid in this class, as it is a generic low level class, and should not be aware about the logic implemented in the classes calling it. Below would be the methods:

public boolean containsResults(String query, String[] params) -> The params we have created as a String array, which will have both user name and password. If the values required for the where clause of the SQL query are of different datatype, we might need a different approach. This method is invoked by the LogonAction

private Connection getConnection() -> Internally used to get a Connection object for the DB call.

private void closeConnection(Connection conn) – Closes an open connection after the query has been executed.

Development

We will first start by adding a struts.xml for our project. In eclipse, you will have to create a folder ‘classes’ under \workspace\Struts2LoginApp\WebContent\WEB-INF\ and add struts.xml in that folder.

Below is the struts.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE struts PUBLIC
   "-//Apache Software Foundation//DTD Struts Configuration 2.0//EN"
   "http://struts.apache.org/dtds/struts-2.0.dtd">
<struts>
<constant name="struts.devMode" value="true" />
   <package name="loginapp" extends="struts-default">
     
      <action name="logon" 
            class="com.tech_freaks.action.LogonAction" 
            method="execute">
            <result name="success">/Home.jsp</result>
            <result name="input">/Login.jsp</result>
            <result name="error">/Login.jsp</result>
      </action>
   </package>
</struts>

 The configuration file is straightforward and easy to understand. <constant> is setting the project in a development mode. This will give helpful errors in the logs / screen while debugging issues.

<package> is applicable when we divide the configure into multiple logical parts instead of having all action definition inside single file.
Let us take a careful look at the entry for <action> and ensure all mapping components are named to match the entries in here.

€¢ €œname€ attribute needs to be mapped to the <form action=€â‚¬> in the Login.jsp. This is how struts knows which Action class to invoke. We leave the method to €œexecute€ as our Action class will extend ActionSupport class and override the execute method.
€¢ €œclass€ should be fully qualified class name of the Action class.
€¢ When the Action executes, we can configure different JSPs / views to invoke after completion. The String value returned from Action.execute() method should match the €œname€ attribute in one of the €œresult€. However, it is important to note that, if we are using struts validation framework, we need to have an <result> with name=€input€. The framework will redirect to the view configured for <result> name=”input” with the error message. If you do not have entry for €œinput€ you will see an error saying that €œNo result defined for action and result input – action €“€œ

Next, we take a look at the LogonAction.java.
As we plan to validate for empty fields, we need to extend the ActionSupport class. Once extended, we need to override the methods execute() and validate(). The next thing is we need to have instance variables for user fields in the Login.jsp. Note that the instance variable (mainly the getter/ setters for the instance variables) must match exactly to the €œname€ attribute on the <input> fields on the JSP. This is also a difference between struts 1.1 were we had separate ActionForms which had the fields mapped to the <input>.
The complete java code is below

 package com.tech_freaks.action;

import java.util.HashMap;
import java.util.Map;
import com.opensymphony.xwork2.ActionContext;
import com.opensymphony.xwork2.ActionSupport;
import com.opensymphony.xwork2.util.ValueStack;
import com.tech_freaks.utils.DBHelper;

public class LogonAction extends ActionSupport {
	
	public static final String LOGIN_SUCCESS = "success";
	public static final String LOGIN_FAILED = "error";
	
	private static final String LOGIN_QUERY = "select * from users where user_name=? and password=?";
	
	private String userName;
	private String password;

	public String getUserName() {
		return userName;
	}

	public void setUserName(String userName) {
		this.userName = userName;
	}

	public String getPassword() {
		return password;
	}

	public void setPassword(String password) {
		this.password = password;
	}

	@Override
	public String execute() throws Exception {
		ValueStack stack = ActionContext.getContext().getValueStack();
		String[] params = new String[] {userName, password};
		DBHelper dbHelper = new DBHelper();
		
		boolean isSuccess = dbHelper.containsResult(LOGIN_QUERY, params);
		if(isSuccess)
			return LOGIN_SUCCESS;
		else {
			Map<String, Object> context = new HashMap<String, Object>();
			context.put("errorMsg", new String("Invalid user name or password. Try again.")); 
			stack.push(context);
			return LOGIN_FAILED;
		}
	}

	@Override
	public void validate() {
		 if (userName == null || userName.trim().equals(""))
	      {
	         addFieldError("userName","Please enter a user name to login with.");
	      }
		 if (password == null || password.trim().equals(""))
	      {
	         addFieldError("password","Please enter a password.");
	      }
	}
}
 

Let”s take a close look at validate(). Use the addFieldError() method with the first parameter matching the field name which has the validation error. Again, this name needs to exactly match with <input> name attribute in the JSP. If we use the struts taglib in the JSP, it also takes care of showing the error message for each field inline to the field.

Next, we take a look at the execute() method. It is fairly straightforward. It calls the DBHelper class to verify in the database, if a record exists for the supplied userName and password combination. Based on the result, it returns €œsuccess€ or €œerror€. The value returned needs to match with the <result> name attribute defined in the struts.xml which we examined before. The appropriate view will be displayed.

For the failed login attempt, we have to go back to the login page and display an error message. We use this requirement to explain ActionContext and ValueStack concept in struts 2. The ActionContext has access to variables like session, request params etc. It also contains a value stack. You can put objects into value stack. As the name suggests, it is a stack with push, pop, peek methods in it. In the JSP, when you want to access the variables in the ActionContext using the struts taglib, you use #<attribute> say #session. However, for accessing value stack, we do not have to use the #. As you can see, we are pushing a Map with key errorMsg into the value stack, with value containing the error message which needs to be displayed to the user on failed login attempt.

Login.jsp
This page is modified from our previous version of Login.jsp. The main change is the struts taglib usage. We use the <s:property> to pull the attribute from the value stack which we previously discussed. This is used where the login attempt fails. We have used <s:form>, <s:textfield> etc. They help giving inline errors for each field as per the login in the validate() method.

Below is the complete Login.jsp

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
   pageEncoding="ISO-8859-1"%>
<%@ taglib prefix="s" uri="/struts-tags"%>
<html>
<head>
<title>
Tech-freaks.com - Struts2 - SimpleLogin App Login Page
</title>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
</head>
<body>
<s:property value="errorMsg"/><br/>
<s:form method="POST" action="logon">
	<s:textfield  name="userName" label="User Name"/>
	<s:password  name="password" label="Password" />
	<s:submit name="submit" label="Submit" align="center" />
		
</s:form>
</body>
</html>

Home.jsp
Simple page, which when requested due to successful login, fetches the €œusername€ field and displays the welcome message. Below is the complete JSP

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><%@page
	language="java" contentType="text/html; charset=ISO-8859-1"
	pageEncoding="ISO-8859-1"%>
<%@ taglib prefix="s" uri="/struts-tags"%>
<html>
	<head>
		<title>
		Tech-freaks.com - Struts 2 SimpleLogin App Home Page
		</title>
		<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
	</head>
	<body>
	 	<h1> Welcome to Tech-freaks.com - Struts2 Simple Logon App, <s:property value="userName"/></h1>
	</body>
</html>

Testing

Once you have all the components put in, right click the project in Eclipse and Export as WAR. Drop the generated WAR in <Tomcat>\webapps folder and start Tomcat. Verify Tomcat logs, if the deployment of the WAR was successful. 
Test by accessing the URL : http://localhost:8080/Struts2LoginApp/Login.jsp (Modify the port and context name as per your customization)
Run and verify each scenario mentioned in the requirement.

Congratulations, on completing your first “Non-Hello World” project using struts 2!

You can download the complete Eclipse project from Github by clicking the link below.

Download complete Struts 2 Login App Source code from Github

Requirement

• Create a Registration Form to enter user name, password, first name, last name, email address, secret question, answer and register a new user to system.
• Validate if the user name and password is not blank. Display error message to user, if either of them is blank.
• Validate if the user name already exists and display an error message to the user
• In the event of an error, ensure that registration form is displayed again with all prefilled information
• On successful registration, set the user name in session and forward to home page.

Pre-requisites

• MySQL and Tomcat is already installed
• You have basic knowledge of MySQL to create tables and insert / update records
• You have basic knowledge of Java and J2EE web applications
• Have your favorite IDE for developing / reviewing code
• This workshop adds the registration functionality to the previous article Simple Login Application. Reviewing that article prior to this article would help.

Concepts Covered

• JDBC connectivity to MySQL from Servlet
• Simple application of JSTL tag
• Using request attribute and request parameters
• Basic error handling concepts

NOTE: Don’t even think of using the code as is for a production environment
This is just for education purpose but you can take inspiration from this code.

Analysis and Design

We start with the Simple Login application article as a baseline. First, we need to update the database to add more fields and constraints. We will make the user_name field as the primary key for the USERS table.

We create a new class for database connectivity code unlike in the simple login application and refactor the connectivity code out of the Servlet code. This is a good design practice to keep the servlet layer uncoupled to the database code layer. We would still following the MVC design model.

Below is a list of elements identified as part of the design:

• com.techfreaks.db.util.ConnectionUtil – Utility class for getting connection, closing connection and executing insert / update query
• com.techfreaks.registration.servlet.RegistrationServlet – This servlet is the controller. It calls the ConnectionUtil and forwards to the home page or Registration page
• RegistrationForm.jsp – UI page with fields for registration form
• Home.jsp – UI page picked up after successful registration. It is the same page taken from Login Application
• web.xml – Deployment Descriptor in which the servlet is registered.

Development

Setup the database:

We alter the USERS table to add the new fields. However, for quick reference we provide the create USERS table DDL. You will have to drop and recreate the table using the below DML, if you already have an USER table. You could very well manually edit the existing table based on the below DDL.

CREATE TABLE `users` (
	`first_name` VARCHAR(100) NOT NULL,
	`last_name` VARCHAR(100) NOT NULL,
	`email_address` VARCHAR(50) NOT NULL,
	`secret_question` VARCHAR(200) NULL DEFAULT NULL,
	`secret_answer` VARCHAR(200) NULL DEFAULT NULL,
	`user_name` VARCHAR(100) NOT NULL,
	`password` VARCHAR(100) NOT NULL,
	PRIMARY KEY (`user_name`)
)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB;

A screenshot of the development environment is provided below. Make sure the mysql drivers, jstl libraries and tlds are all available. For making your life easy, we have included a hot deployable WAR file at the end of the workshop.

Coding and Unit testing:

First, we would look at the ConnectionUtil. Below are the methods we would need for the ConnectionUtil. Note that we create this class as a static class (A class with static methods). 

– getConnection()
– closeConnection()
– executeQuery(String strQuery)

If you have read the article about simple login application, you would have already noticed thatt we just refactored the first two methods out of the LoginServlet. We prevent the code to be spread across multiple servlets this way. It is used to get a SQL Connection object and to close an open connection respectively.

In this example, we would use java.sql.Statement instead of java.sql.PrepareStatement which we used in LoginServlet, to spice things up. We need to pass the complete query to the Statement object to execute a query. The executeQuery can take an insert / update query and execute it. It would internally call getConnection() and closeConnection() to fetch and close a connection.

Below is the method executeQuery:

public static void executeQuery(String strQuery) throws Exception {
		Connection conn = null;
		
		try {
			conn = getConnection();
			Statement stmt  = conn.createStatement();
			stmt.executeUpdate(strQuery);
			
		} catch (SQLException sqle) {
			System.out.println("SQLException: Unable to execute query : "+strQuery);
			throw sqle;
		} catch (Exception e) {
			System.out.println("Exception: Unable to execute query: "+strQuery);
			throw e;
		} finally {
			closeConnection(conn);
		}
	}

We also moved the static variables like database name, db user name and db password from the servlet to the ConnectionUtil.

        private static final String DBNAME = "tf_loginappdb";
	private static final String DB_USERNAME = "root";
	private static final String DB_PASSWORD = "admin*";

If your database has different dbname, username or password, you will need to update this and recompile you java file for the application to be able to connect to MySQL database.

Next, we would look at the code in the RegistrationServlet. It gets all the fields from the RegistrationForm in the request parameters. Below are the methods which we would add to this servlet.

– doPost()
– validateData()
– setRequestAttributes()
– generateInsertQuery()

The doPost is the entry point to the servlet for html form POST. We have all the controller logic in here. The other methods are in a way, do not do the controller work. We use them for simple tasks which do not warranty creating other classes in our case. However, in a real life application, if the task is complex we might want to create new classes to delegate the task.

validateData() is used to implement the requirement that the userName and password cannot be empty. It validates and returns a boolean indicating success or failure of the validation.

generateInsertQuery() fetches the request parameters and generates the insert query which will be passed on to the executeQuery method of the ConnectionUtil. Below is the code for the method:

 private String generateInsertQuery(HttpServletRequest request) {
		String strUserName = request.getParameter("userName");
		String strPassword = request.getParameter("password");
		String strFirstName = request.getParameter("firstName");
		String strLastName = request.getParameter("lastName");
		String strEmail = request.getParameter("email");
		String strSecretQuestion = request.getParameter("secretQuestion");
		String strSecretAnswer = request.getParameter("secretAnswer");

		StringBuffer strQuery = new StringBuffer(INSERT_QUERY_START);
		strQuery.append(strFirstName);
		strQuery.append("', '");
		strQuery.append(strLastName);
		strQuery.append("', '");
		strQuery.append(strEmail);
		strQuery.append("', '");
		strQuery.append(strSecretQuestion);
		strQuery.append("', '");
		strQuery.append(strSecretAnswer);
		strQuery.append("', '");
		strQuery.append(strUserName);
		strQuery.append("', '");
		strQuery.append(strPassword);
		strQuery.append("')");
		
		System.out.println("Insert query : "+strQuery.toString());
		
		return strQuery.toString();

	}
 

setRequestAttributes() is used to implement the requirement about maintaining the data entered in the registration form in the even of an error. Do you remember banging your head against the keyboard after filling up a long registration form and clicking submit, only to find the form reloading with an error message and all the data you entered spending hours, boom! gone!

Instead of the classic get each parameter and set to attribute, we would just do a generic loop through. This would keep the code small. Imagine your registration form is 100 fields instead, the below code will not require any modification. Below is the code:

	private void setRequestAttributes(HttpServletRequest request) {
		Enumeration  enumKeys =  request.getParameterNames();
		while(enumKeys.hasMoreElements()) {
			String key  = enumKeys.nextElement();
			request.setAttribute(key, request.getParameter(key))  ;
		}
	} 

The doPost() method is provided below. It has inline comment to explain key control logic. Once the insert in the database is successful, we set the userName is session and redirect to Home.jsp.

The error messages are managed by putting a request attribute on error situation and forwarding back to RegistrationForm. You will see the difference between the redirect and forward. You will see that the request attributes are available and also the URL of the servlet remains the same after the request is forwarded to the JSP. Also, note how we utilize the primary key duplicate error message and set the error message. We use the same pattern to set error message, set the request attributes and forward to registration page on all the error conditions.

protected void doPost(HttpServletRequest request, HttpServletResponse 
			response) throws ServletException, IOException {
		
		String strUserMsg = null;
		HttpSession session = request.getSession();
		RequestDispatcher reqDisp =  request.getRequestDispatcher(REGISTRATION_PAGE);
		
		try {
			//Check if data is valid
			if(validateData(request)) {
				ConnectionUtil.executeQuery(generateInsertQuery(request));
				System.out.println("Insert into database successful");
				session.setAttribute("userName", request.getParameter("userName"));
				response.sendRedirect(getServletContext().getContextPath()+HOME_PAGE);
			} else {//If data is invalid
				strUserMsg = "User Name and Password cannot be empty";
				setRequestAttributes(request);
				request.setAttribute("userMsg", strUserMsg);
				reqDisp.forward(request, response);
			}
			
		} catch(SQLException sqle ) {
			System.out.println("Unable to register user: "+sqle.getMessage());
			//Check if we are getting duplicate key exception on userName
			if(sqle.getMessage().indexOf("Duplicate entry")!=-1) {
				System.out.println("User already exists");
				strUserMsg = "User name "+request.getParameter("userName")+" already " +
						"exists. Please try another user name.";
			} else { //If other SQLException than dup key exception
				strUserMsg = "Unable to register user "+request.getParameter("userName")+
				". Please try again later.";
			}
			setRequestAttributes(request);
			request.setAttribute("userMsg", strUserMsg);
			reqDisp.forward(request, response);

		} catch(Exception e) {//If it goes into Exception other than SQLException
			System.out.println("Unable to register user: "+e.getMessage());
			strUserMsg = "Unable to register user "+request.getParameter("userName")
			+". Please try again later.";
			setRequestAttributes(request);
			request.setAttribute("userMsg", strUserMsg);
			reqDisp.forward(request, response);

		}
		
		

	}

Next, let us look at our RegistrationForm.jsp. We highlight small snippet of code here

<p><font color="#ff0000"><c:out value="${userMsg}"/></font></p>
<form name="frmRegistration" method="post" action="<c:out value="${pageContext.servletContext.contextPath}" />/servlet/RegistrationServlet">
<table border="1">
<tbody>
<tr>
<td>First Name</td>
<td><input type="text" name="firstName" value ="<c:out value="${firstName}"/>" size="20"></td>

• JSTL is used display the userMsg which we set in the request attribute in our servlet.
• JSTL is used to pick the contextPath from servletContext
• JSTL is used to display user entered data in the event of an error

We picked up the Home.jsp from the simple login application. You can get the complete WAR file to the simple registration application by clicking SimpleRegistrationApp.war.

You just need to hot deploy SimpleRegistrationApp.war to webapps folder in the Tomcat program file and it would explode. Verify the application by accessing http://localhost:8080/SimpleRegistrationApp/RegistrationForm.jsp and run some tests to figure out how it works for you.

Feel free to leave a comment below, if you run into any issues. Enjoy!

This workshop is for you, if:

• You are looking for a data structure to store your data for easy javascript access
• You have seen JSON structure but do not know, how to access it using javascript

This workshop is not for you, if:

• You have no idea about client side scripting

This workshop intends to provide hands on example on creating JSON structure and accessing the contents of the JSON using Javascript.

‘JSON’ the new kid in town

JSON which stands for JavaScript Object Notation is the new buzz word and will definitely look good in your resume! But, how is JSON useful from a technology perspective.

• JSON can store data in the client side, where data security is not important.
• JSON can be used for data exchange just like XML. But, why not XML itself? JSON comes handy when the exchanged data needs to be handled in the client side. JSON structure is easy to be manipulated in client side using javascript.

Workshop Requirement

Let us say, we need to maintain the summary of contents of this website. We want the data not to be maintained in database. We need to get information from a data structure using Javascript and of course, this data is not sensitive. The website contains, two sections (while writing this article), Java Programming and Stock Market. Each section contains categories and each category contains articles. Based on this data, we would like to see the number of pages in Java category etc.

Meeting JSON flesh and blood

Enough talking about JSON! Let us actually see a JSON structure. The code snippet provided below shows a JSON structure assigned to a Javascript variable. When we use JSON for data exchange, we will write or receive JSON structure starting from braces without the javascript variable. To convert this into a javascript variable, we can use ‘eval’ function, which is not recommended. There are parsers available in JSON.org which converts the JSON structure to javascript variable.

var varSiteData = {
  "sections" : [ //Array
     {//Each element in array, ie. Section is a structure not a simple attribute
      "name": "Java Programming", //this is a simple attribute
      "categories": [
      {
         "name":"Java Basics",
         "articles": [
         {
            "name" : "Java Hello World",
            "pages" : "1",
            "author" : "Tech Programmer"         
         },
         {
            "name" : "Java Hello World using Eclipse IDE",
            "pages" : "5",
            "author" : "Tech Programmer"         
         },
         ...
         ...
       ]
      },
      ...
      ...
     ]
    },
    ...
    ....
   ]
 }

Let us try to understand the syntax. JSON always start and end with curly braces. They contain attributes. The value of each attribute can be simple string, a structure (which internally is composed of many simple string attributes), array of simple string or array of structures. Here “section” is an attribute. How many sections we have in the site? Two, Java programming and Stock market. This calls for an array. Arrays are defined by square brackets. Now, each section contains categories. Also, each section needs a name to be stored. So, we have added an attribute by “name”. Attribute “name” is a simple string. Other attributes shown work in the same way. Click here to download the complete JSON structure stored in JS file.

• An attribute name or key cannot start with numeric value. Although this is not a rule written anywhere, when the key or attribute name starts with or is a numeric value, the javascript which parse through the JSON does not understand the key. A simple workaround to resolve this issue is to append a constant string before the numeric value. Example: “str_25”

 One more important thing is, if for an attribute, the value needs to be multiple attributes but not an array, then it is achieved using curly brackets only. A simple snippet which shows the attribute structure for Alphonsa mango:

"alphonsa" : {
  "type": "mango",
  "taste": "sweet",
  "color": "yellow"
}

Well, that’s that. We got started with JSON. But, let us try to have a quick look, why JSON is compared with XML. See the above JSON structure in an XML format.

<?xml version="1.0" encoding="ISO-8859-1"?>
<sections>
  <section>
     <name>Java Programming</name>
     <category>
        <name>Java Basics</name>
        <article>
           <name>Java Hello World</name>
           <pages>1</pages>
           <author>Tech Programmer</author>
        </article>
        <article>
           <name>Java Hello World using Eclipse IDE</name>
           <pages>5</pages>
           <author>Tech Programmer</author>
        </article>
       ...
       ...
     </category>
  </section>
..
</sections>

Validating JSON structure

Just like we validate XML structures using Internet Explorer and other tools for well-formedness, there are certain JSON viewers available, which verify the JSON structure and provide details regarding any missing braces etc. JSON Viewer is one such hand helpful too.

Javascript and JSON

Now, let us see how Javascript can access different elements within the JSON structure. We will try to calculate the total number of pages in ‘Stock Market’ and ‘Java Programming’ section. The HTML page includes the JSON data from a JS file. The javascript present in the HTML page iterates through the JSON structure to determine the result.

See the below javascript function, with comments explaining how the JSON structure is iterated through.

function getSectionTotalPages(section) {

 //entry point into JSON variable..Accessing sections attribue, which is an array
 var arrSections = varSiteData.sections;
 var totalPages = 0;
 //loop through each of the sections
 for(var i=0;i<arrSections.length;i++) {
  var currSection = arrSections[i];
  //we get the section for which we want to calculate the page total
  if(currSection.name==section) {
   //Now into category array from section
   var arrCategories = currSection.categories;
   
   //loop through the category array
   for(var j=0;j<arrCategories.length;j++) {
    var currCategory = arrCategories[j];
    //For each category, get the article list (array)
    var arrArticles = currCategory.articles;
    //loop through each article
    for(var k=0;k<arrArticles.length;k++) {
     var currArticle = arrArticles[k];
 
     //for each article, get the # of pages and add it to total
     totalPages = totalPages + parseInt(currArticle.pages);
    }
   }
   //This method works for one section, so break the loop, if the section is found
   break;
  }
 }
 return totalPages;
}

The complete HTML containing the javascripts is given below. On clicking the compare button, the compareSectionPages function is called. It internally,calls getSectionTotalPages function twice, for getting pages for 'Java Programming' section and 'Market Analysis' section.

<html>
<head>
  <title>JSON and Javascript example</title>
  <script type="text/javascript" src="website_json.js" ></script>
  <script language="javascript" type="text/javascript">
  function getSectionTotalPages(section) {
 var arrSections = varSiteData.sections;
 var totalPages = 0;
 for(var i=0;i<arrSections.length;i++) {
  var currSection = arrSections[i];
  if(currSection.name==section) {
   var arrCategories = currSection.categories;
  for(var j=0;j<arrCategories.length;j++) {

    var currCategory = arrCategories[j];
    var arrArticles = currCategory.articles;
    for(var k=0;k<arrArticles.length;k++) {
     var currArticle = arrArticles[k];
     totalPages = totalPages + parseInt(currArticle.pages);
    }
   }
   break;
  }
 }
 return totalPages;
}

  function compareSectionPages() {
   var totalJavaPages = getSectionTotalPages('Java Programming');
 var totalMarketPages = getSectionTotalPages('Market Analysis');
   var result = "<table border='1'><tr><td>Java Programming</td><td>Market Analysis</td>

</tr><tr><td>"+totalJavaPages+"</td><td>"+totalMarketPages+"</td></tr></table>";
   document.getElementById('result').innerHTML = result;
  }

  </script>
</head>
<body>
  <h2>Welcome to JSON with javascript workshop!</h2>
  <p>
  <input type="button" name="button1" value="Compare Section pages" onclick="compareSectionPages();">

  <p><p>
  Result: <div id="result"></div>
</body>
</html>

 And the result is 17 to 12. Java programming wins by 5 goals!

The author of the ‘Market analysis’ section may feel bit hard done by.  Some articles missed out were missed out. ‘Technical Analysis’ and ‘Market Summary’ articles were not considered. We added every article inside the category, but these two articles are not maintained at category level. They are maintained in section level.

Well, change is inevitable in the JSON structure! However, the flexibility of JSON will cover up for it. But, the change in data structure will cause a change to javascript code to calculate the total pages in section.

Let us get this thing fixed. For the JSON fix, a little thought will make it clear that adding articles inside section structure will be similar to articles maintained inside categories. The below code snippet shows the portion of the changed JSON:

 ...
      {
      "name":"Market Analysis",
      "articles": [
       {
          "name": "Market Summary",
          "pages": "1",
          "author" : "Market Analyst"        
       },
       {
          "name": "Tech Analysis",
          "pages": "1",
          "author" : "Market Analyst"        
       }
      ] 
      ....
     .....

The change in  getSectionTotalPages will be bit more challenging. Now, in the code we have to deal with one section, which has articles array and another which does not. Also, this method needs to be generic enough to handle any new sections added to the website in the future. Some of the section may have articles inside it and some may not. Many times, while using javascript we see the ‘undefined’ variable. We will have to use this, to check if articles array is present directly within sections.

The below modified function getSectionTotalPages will do the trick. See the comments explaining the main changes and the typeof function used to good effect.

function getSectionTotalPages(section) {
 var arrSections = varSiteData.sections;
 var totalPages = 0;
 for(var i=0;i<arrSections.length;i++) {
  var currSection = arrSections[i];
  if(currSection.name==section) {
   //Check if articles inside section is undefined (does not exist)
   //If not undefined, then section as articles like Market Analysis section
   if(typeof(currSection.articles)!='undefined') {

    var arrSecArticles = currSection.articles;
    for(var j=0;j<arrSecArticles.length;j++) {
     var currSecArticle = arrSecArticles[j];
     totalPages = totalPages + parseInt(currSecArticle.pages);

    }
   }
   var arrCategories = currSection.categories;

   for(var j=0;j<arrCategories.length;j++) {
    var currCategory = arrCategories[j];
    var arrArticles = currCategory.articles;
    for(var k=0;k<arrArticles.length;k++) {
     var currArticle = arrArticles[k];
     totalPages = totalPages + parseInt(currArticle.pages);
    }
   }
   break;
  }
 }
 return totalPages;
}

Update the HTML code contents with the above updated function. Click here to download the updated JSON code. Place the HTML and JS file in the same directory. Access the HTML page from browser and then click the ‘Compare Section pages’ button to view the result table. ‘Java Programming’ still wins by a margin of 3!

Welcome to the whole new world of client side scripting with JSON!