This article originally appeared in the December 2012 issue of GroovyMag.
Some different ways that Groovy makes interacting with the web easier
One of the major benefits of Groovy is how it simplifies some of the common scenarios we deal with in Java. Complex code with conditionals, error handling and many other concerns can be expressed in a very concise and easily understandable fashion. This article will touch on some convenient Groovy-isms related to interacting with content over HTTP. First we’ll look at some of the syntactic sugar added to the standard Java classes that simplify GET and POST requests, and then we’ll take a look at how the HTTPBuilder module provides a DSL for using the HttpClient library.
The test project
In order to provide an environment for putting up a website and demonstrating various HTTP requests, we’ll be using the Gradle Jetty plugin and some simple Groovlets. The full source code is available at https://github.com/kellyrob99/groovy-http and I hope you’ll clone a copy to take a closer look. The simple index page contains the ‘hello world’ content shown in Listing 1.
[sourcecode language=”xml”]
<!DOCTYPE html>
<html>
<head>
<title>Groovy HTTP</title>
</head>
<body>
<p>hello world</p>
</body>
</html>
[/sourcecode]
Listing 1: Our ‘hello world’ index page used for testing
We’ll start with the simplest available methods for interacting with HTTP using Groovy and no additional library support.
Groovy methods added to String and URL
The DefaultGroovyMethods class provides a couple of very handy methods to enhance the default operation of the String and URL classes. In particular for String we have a new toURL() method and, for URL, the text property. In addition, the URL class is enhanced with convenience methods for working with associated InputStream and OutputStreams.
String.toURL()
This is a small gain as all you’re really doing is avoiding a call to new URL(String spec). The difference in keystrokes isn’t large but, combined with some other MetaClass benefits of Groovy, it can be very helpful for creating fluent and easily understandable code.
URL.text()
This seemingly small addition to the API of the URL class abstracts away a lot of the usual boilerplate involved in streaming content over a URLConnection. Underneath the hood is a very sensible implementation that buffers the underlying connection and automatically handles the closing of all resources for you. For most use cases the default behaviour is likely to be sufficient but, if not, there are overloaded URL.text(String charset) and URL.text(Map parameters, String charset) methods that allow for modification and handle more specifics of the connection configuration.
The one line invocation in Listing 2 demonstrates how to load an html page, returning the raw html as a String.
[sourcecode language=”groovy”]
String html = ‘http://localhost:8081/groovy-http’.toURL().text
[/sourcecode]
Listing 2: One liner to initiate an HTTP GET request for an html page
There’s still a lot that could go wrong using this shorthand syntax for an HTTP request, as several exceptions might be thrown depending on whether or not the url is correctly formatted, or if the content specified doesn’t exist. The Spock test shown in Listing 3 exercises both of these conditions. Note that a 404 response will result in a FileNotFoundException.
[sourcecode language=”groovy”]
@Unroll("The url #url should throw an exception of type #exception")
def "exceptions can be thrown converting a String to URL and accessing the text"() {
when:
String html = url.toURL().text
then:
def e = thrown(exception)
where:
url | exception
‘htp://foo.com’ | MalformedURLException
‘http://google.com/notThere’ | FileNotFoundException
}
[/sourcecode]
Listing 3: Spock test showing some possible failure conditions for our GET request
For comparison let’s take a look at what the same GET request looks like using a URL in Java, shown in Listing 4.
[sourcecode language=”java”]
URL html = new URL(‘http://localhost:8081/groovy-http/index.html’);
URLConnection urlConnection = html.openConnection();
BufferedReader reader = new BufferedReader(
new InputStreamReader(urlConnection.getInputStream()));
StringBuffer response = new StringBuffer();
String inputLine;
while ((inputLine = reader.readLine()) != null)
{
response.append(inputLine)
}
reader.close();
[/sourcecode]
Listing 4: The Java version of reading from a URLConnection (based on the canonical example from Oracle.com)
There’s still no error handling in place in the Java version which is obviously a much more verbose way to load the same data.
POST with URL streams
Similarly to simplifying GET requests, executing a POST using Groovy can take advantage of some of the enhancements to common Java classes. In particular, simplified stream handling allows for tight, correct and expressive coding. Listing 5 shows a Spock test configuring the URLConnection, POSTing some data and reading back the result from the connection.
[sourcecode language=”groovy”]
private static final String POST_RESPONSE = ‘Successfully posted [arg:[foo]] with method POST’
def "POST from a URLConnection"() {
when:
final HttpURLConnection connection = makeURL(‘post.groovy’).toURL().openConnection()
connection.setDoOutput(true)
connection.outputStream.withWriter { Writer writer ->
writer << "arg=foo"
}
String response = connection.inputStream.withReader { Reader reader -> reader.text }
then:
connection.responseCode == HttpServletResponse.SC_OK
response == POST_RESPONSE
}
[/sourcecode]
Listing 5: POST request using Groovy and a URLConnection
Notice that we don’t have to explicitly cast the connection to HttpUrlConnection in order to get the responseCode back, and that we don’t have to explicitly close any of the streams used. Also, we don’t need to create local variables for the Reader/Writer object as we would have to in Java; similarly no calls to ‘new’ are required, as Object creation is all hidden behind the convenience methods. The equivalent Java code requires four calls to new and two to close(), as well as much more involved code for extracting the result. The canonical example of how to do this in Java can be seen on http://docs.oracle.com/javase/tutorial/networking/urls/readingWriting.html
Note that you can also parse response content very easily using the XmlSlurper / XmlParser and JsonSlurper classes included in the standard Groovy distribution.
HttpClient and HTTPBuilder make things even easier
The reality is that in most modern Java applications developers have some nice alternatives to directly working with URL and URLConnection objects for working with HTTP. One of the more popular libraries available is HttpClient and its successor HttpComponents. Wrappers for all of the HTTP verbs are provided which simplifies configuration, execution and consumption of responses. Listing 6 shows a Spock test using HttpClient and mirroring our prior GET examples.
[sourcecode language=”groovy”]
def "HttpClient example in Java"() {
when:
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(makeURL("helloWorld.groovy"));
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpclient.execute(httpget, responseHandler);
then:
responseBody == HELLO_WORLD_HTML
}
[/sourcecode]
Listing 6: HttpClient GET example
This can be further reduced if there is no need for keeping the intermediate variables around. In fact, we can get it down to the single line shown in Listing 7.
[sourcecode language=”groovy”]
String response = new DefaultHttpClient().execute(new HttpGet(makeURL("helloWorld.groovy")), new BasicResponseHandler())
[/sourcecode]
Listing 7: HttpClient GET one-liner
This is obviously a lot easier on the eyes and very clear in intent. The HttpClient library also has convenience mechanisms for declaring common behaviour across connections, an API for providing custom response parsing implementations and automatic handling for (most of) the underlying resource streams and connections. For those of us using Groovy, there’s a nice wrapper for HttpClient called HTTPBuilder that adds a DSL-style configuration mechanism and some very nice features in terms of error handling and content parsing. Listing 8 shows our standard GET example again, this time working against an object called http assigned from new HTTPBuilder(Object uri). Note that we’re using Groovy’s multiple assignment feature to return and assign multiple values from our Closure.
[sourcecode language=”groovy”]
def "GET with HTTPBuilder"() {
when:
def (html, responseStatus) = http.get(path: ‘helloWorld.groovy’, contentType: TEXT) { resp, reader ->
[reader.text, resp.status]
}
then:
responseStatus == HttpServletResponse.SC_OK
html == HELLO_WORLD_HTML
}
[/sourcecode]
Listing 8: Spock test showing Groovy HTTPBuilder GET support
If you noticed in Listing 8 I explicitly set the request with contentType: TEXT, it’s because HTTPBuilder by default provides automatic response content type detection and parsing. Since I’m requesting an xml document, HTTPBuilder can automatically parse the result with Groovy’s XmlSlurper. HTTPBuilder can also detect that it is an html page and pass the response through NekoHTML first to ensure that you’re working with a well-formed document. Listing 9 shows the slight difference in how we could interact with the parsed response content and the reader in our Closure from Listing 8 is quietly replaced with a GPathResult referring to the parsed content.
[sourcecode language=”groovy”]
def "GET with HTTPBuilder and automatic parsing"() {
when:
def (html, responseStatus) = http.get(path: ‘helloWorld.groovy’) { resp, reader ->
[reader, resp.status]
}
then:
responseStatus == HttpServletResponse.SC_OK
html instanceof GPathResult
html.BODY.P.text() == ‘hello world’
}
[/sourcecode]
Listing 9: automatic detection and parsing of xml
It’s unlikely that you’re going to be parsing a lot of html this way but with the abundance of xml services available nowadays automated parsing can be very helpful. The same applies for JSON and if we give a hint as to the contentType we can get back a parsed JSONObject when interacting with such services as shown in Listing 10.
[sourcecode language=”groovy”]
def "GET with HTTPBuilder and automatic JSON parsing"() {
when:
def (json, responseStatus) = http.get(path: ‘indexJson.groovy’, contentType: JSON) { resp, reader ->
[reader, resp.status]
}
then:
responseStatus == HttpServletResponse.SC_OK
json instanceof JSONObject
json.html.body.p == ‘hello world’
}
[/sourcecode]
Listing 10: automatic parsing of JSON responses
The HTTPBuilder module also has some convenience methods for handling failure conditions. By allowing for specifying both default failure handlers and specific behaviour for individual requests you’ve got lots of options at your disposal. Listing 11 shows how to define a default failure handler that simply traps the response code. Note that the Closure used for hading GET response is never run since in this case the page we’re requesting results in an HTTP 404 Not Found response code.
[sourcecode language=”groovy”]
def "GET with HTTPBuilder and error handling"() {
when:
int responseStatus
http.handler.failure = { resp ->
responseStatus = resp.status
}
http.get(path: ‘notThere.groovy’, contentType: TEXT) { resp, reader ->
throw new IllegalStateException(‘should not be executed’)
}
then:
responseStatus == HttpServletResponse.SC_NOT_FOUND
}
[/sourcecode]
Listing 11: Defining a failure handler with HTTPBuilder
POSTing data with HTTPBuilder is also very straightforward, requiring only an additional body parameter as shown in Listing 12.
[sourcecode language=”groovy”]
def "POST with HTTPBuilder"() {
when:
def (response, responseStatus) = http.post(path: ‘post.groovy’, body: [arg: ‘foo’]) { resp, reader ->
[reader.text(),resp.status]
}
then:
responseStatus == HttpServletResponse.SC_OK
response == POST_RESPONSE
}
[/sourcecode]
Listing 12: POST using HTTPBuilder
HTTPBuilder also provides some more specific abstractions for dealing with certain scenarios. There’s RESTClient for dealing with RESTful webservices in a simplified manner, there’s AsyncHTTPBuilder for asynchronously executing requests and for the Google App Engine, which doesn’t allow socket based connections, there’s the HttpURLClient which wraps HttpUrlConnection usage.
Conclusion
Hopefully this has given you a taste for what Groovy can do to help you with HTTP interactions and gives you some ideas for making your own HTTP client applications a bit Groovier.
More reading
- http://docs.oracle.com/javase/tutorial/networking/urls/readingWriting.html
- http://stackoverflow.com/questions/2793150/how-to-use-java-net-urlconnection-to-fire-and-handle-http-requests
- The HTTPBuilder website http://groovy.codehaus.org/modules/http-builder/
- The HttpComponents website http://hc.apache.org/index.html
- The source code that goes along with this article on github at https://github.com/kellyrob99/groovy-http. This project includes the Gradle wrapper so you should be able to just clone the repository and start using it without installing any additional software(other than Java, of course). You can run all of the tests with ./gradlew clean build and you can start the webserver with ./gradlew jettyRun
Next month we’ll take a closer look at how Groovy can simplify working with a variety of embedded http server alternatives – see you then!
