Bash on Windows 10

Because I work with Linux and Windows based machines for development, I often find myself wishing that I had some of the handy command-line Linux tools available in my Windows environments. Cygwin, PowerShell, and custom Groovy scripts written to emulate Linux tools have helped, but I was pleasantly surprised to recently learn that Bash on Ubuntu on Windows 10 is available. In this post, I briefly summarize some of the steps to make bash available on Windows. More detailed instructions with helpful screen snapshots can be found in the Installation Guide.

The Windows Subsystem for Linux (WSL) is described in the Frequently Asked Questions page as "a new Windows 10 feature that enables you to run native Linux command-line tools directly on Windows, alongside your traditional Windows desktop and modern store apps." This same FAQ page states that enabling WSL "downloads a genuine Ubuntu user-mode image, created by Canonical."

The following are the high-level steps for getting Windows 10 ready to use WSL bash.

1. Verify that Windows 10 installation is 64-bit "system type" and has an "OS Build" of at least 14393.0.
   
2. Turn on "Developer Mode"
   
3. Enable Windows Subsystem for Linux (WSL) ["Turn Windows Features On or Off" GUI]
4. Enable Windows Subsystem for Linux (WSL) [PowerShell Command-line]
   
5. Restart Computer as Directed
6. Run bash from Command Prompt (Downloads Canonical's Ubuntu on Windows)
   
7. Create a Unix User
   

Once the steps described above have been executed, bash can be easily used in the Windows 10 environment. A few basic commands are shown in the next screen snapshot. It shows running bash in the Command Prompt and running a few common Linux commands while in that bash shell.

As a developer who uses both Windows and Linux, having bash available in Windows 10 is a welcome addition.

Related Online Resources

• Everything You Can Do With Windows 10’s New Bash Shell
• What You Need to Know About Windows 10’s Bash Shell
• How to get Bash on Windows 10 with the Anniversary Update
• How to get started with Ubuntu and Bash on Windows 10
• How to install Bash on Windows 10
• Fun with the Windows Subsystem for Linux

Book Review: Penetration Testing with the Bash Shell

This post is my review of the Packt Publishing book Penetration Testing with the Bash shell by Keith Makan. I think it's important to emphasize its subtitle: "Make the most of the Bash shell and Kali Linux's command-line-based security assessment tools." As the main title and sub-title suggest, this book is about penetration testing with Bash using the Kali Linux. The book has approximately 125 pages covering 5 chapters.

The Preface of Penetration Testing with the Bash shell begins with an articulation of why it is important for penetration testers to be familiar with command-line tools. The Preface also provides brief summaries of each of the five chapters in the book and some of the command-line tools covered in each chapter.

Under the "What you need for this book" section of Penetration Testing with the Bash shell's Preface, Makan writes, "The only software requirement for this book is the Kali Linux operating system, which you can download in the ISO format from http://www.kali.org." You don't necessarily need to use Kali Linux as many of the bash examples and described command-line tools are also available with or for other Linux distributions. The "Who this book is for" section of the Preface states, "Command line hacking is a book for anyone interested in learning how to wield their Kali Linux command lines to perform effective penetration testing."

Chapter 1: Getting to Know Bash

The initial chapter explains that "throughout the book, the bash environment or the host operating system that will be discussed will be Kali Linux" and that "Kali Linux is a distribution adapted from Debian." The chapter then goes onto explain with text and examples Bash concepts such as man pages and bash commands such cd, ls, pwd, and find. The chapter also describes using Linux redirection (output and input) and pipes before concluding with a discussion of grep and regular expressions. As the author suggests, this chapter could be generally be skipped by developers familiar with Linux, but would be must-read information for those new to Linux. Although Kali Linux is used for the examples, I didn't see anything in this initial chapter that seemed specific to Kali Linux.

Chapter 2: Customizing Your Shell

The second chapter of Penetration Testing with the Bash shell begins with coverage of terminal control sequences to change appearance of text in the terminal. The chapter builds upon this information to demonstrate customization of the terminal prompt. Chapter 2 also introduces aliases, customizing command history, and customizing tab completion. Like the first chapter, this second chapter covers additional general Bash concepts rather than specifics of penetration testing.

Chapter 3: Network Reconnaissance

Chapter 3 of Penetration Testing with the Bash shell transitions from coverage of Linux and bash to "move on to using the shell and the Kali Linux command-line utilities to collect information about the networks you find yourself in." The chapter examines tools such as whois, dig, dnsmap, Nmap, and arping.

Chapter 4: Exploitation and Reverse Engineering

Penetration Testing with the Bash shell's fourth chapter focuses on reverse engineering and tools that "may enable you to discover memory corruption, code injection, and general data- or file-handling flaws that may be used to instantiate arbitrary code execution vulnerabilities." Tools covered in this chapter include Metasploit (including command-line msfcli, mixing msfcli with bash commands and other command line tools, msfpayload, Meterpreter), objdump, and GDB.

Chapter 5: Network Exploitation and Monitoring

The fifth and final chapter of Penetration Testing with the Bash shell focuses "on the network exploitation available in Kali Linux and how to take advantage of it in the modern bash shell environment." This relatively longer chapter begins with discussion of "Media Access Control (MAC) spoofing" and "Address Resolution Protocol (ARP) abuse." The chapter looks at tools such as macchangeer, ifconfig, arpspoof.

Chapter 5 describes man-in-the-middle (MITM) attacks and provides a detailed introduction to ettercap. The chapter also explains server interrogation and describes tools for Simple Network Management Protocol (SNMP) interrogation (snmpwalk and Metasploit's snmp_enum and snmp_login) and for Simple Mail Transfer Protocol (SMTP) interrogation (smtp-user-enum).

There is a section in Chapter 5 on brute force authentication that demonstrates use of Medusa. The section of Chapter 5 on traffic filtering demonstrates use of TCPDump. SSLyze is demonstrated as a tool for "assess[ing] SSL/TLS implementations" and SkipFish and Arachni are demonstrated as tools for scanning web pages and web applications for vulnerabilities.

General Observations

• The electronic version of Penetration Testing with the Bash shell that I reviewed included numerous colored screen snapshots.
• I liked the "Further Reading" sections at the end of each chapter that provided links to online information with more detail about subjects covered in the chapter.
• The first two chapters of Penetration Testing with the Bash shell have very little information specific to penetration testing but provide an overview of some bash features used in later chapters. The third, fourth, and fifth chapters are heavily focused on penetration testing and build upon the bash basics covered in the first two chapters.
• Penetration Testing with the Bash shell provides background information on various security and assessment concepts as its illustrates the tools available that are related to those concepts.
• There are some awkward sentence structures in Penetration Testing with the Bash shell and several of the sentences are too long for my taste (especially early in the book), but I was generally able to make out the intent of the writing. An example of the occasional awkward but understandable text is on the first page of the first chapter: "Why are discussing the bash shell?"
• Although this book does mention Kali Linux specifically and frequently, the majority of the described tools are available as built-in tools or as tools that can be installed separately with other flavors of Linux. With no or just a a bit of effort, one could run the examples in different Linux implementations. It's also worth pointing out that the author repeatedly mentions that several commands runs with no special effort in Kali Linux because things are run as root. In other Linux implementations, you may need to use sudo to apply these tools.
• Although I am fairly familiar with bash, I still learned a few new things from the first two chapters (primarily the second chapter). I am far less experienced with penetration testing and learned quite a bit from the other three chapters.

Conclusion

Penetration Testing with the Bash shell: Make the most of the Bash shell and Kali Linux's command-line-based security assessment tools outlines how bash in general and Kali Linux in particular provide command-line security assessment tools. The book introduces the tools and how to apply them and explains security-related concepts along the way.

Searching JARs for String (Linux)

I recently thought about writing a Groovy script to search JARs for a specific string, but decided to first look for an alternative rather than writing a script. The alternative needed to be easy to use, freely available, and not bogged down with a load of dependencies. I was glad I looked first, because the Linux-based approach provided by "jan61" satisfied my need nicely. In this blog post, I look at that example more closely.

As documented on the LinuxQuestions.org thread searching contents of .class file within jar (for strings), a single line command in Linux nicely does the job of searching JARs recursively from a given directory for a given String. This search isn't searching for the names of the entries themselves in the JAR, but rather is searching the contents of each searchable file in the JAR. Here is that simple line (the token <stringWithOrWithoutSpacesToFind> represents the string to search for):

Linux Command to Search Contents of .jar Files for Specific String

find . -iname '*.jar' -printf "unzip -c %p | grep -q '<stringWithOrWithoutSpacesToFind>' && echo %p\n" | sh

I like to have this in script form (or as an alias) because that name is easier to remember than typing in that entire command each time. Here is an example script that could be used.

Linux Script Form of Above Command

printf "Searching JARs for string '${1}'...\n"
find . -iname '*.jar' -printf "unzip -c %p | grep -q '${1}' && echo %p\n" | sh

The two versions of the command shown immediately above will work as-is and the rest of this blog post focuses on how the command works. I start analyzing the command from the inside and move outward.

The Linux unzip command "list[s], test[s] and extract[s] compressed files in a ZIP archive." The -c passed to the unzip command "extract[s] files to stdout" and includes the name of the extracted files with that standard output. The %p is associated with the Linux find command. More specifically, %p is a directive to the -printf flag of the find command that directs it to include the found file's name.

Each found file is unzipped and its content directed to standard output where it is piped to a grep command to search for the provided text String. The provided -q parameter specifies "quiet" mode in which nothing is written to standard output and the grep exits immediately with zero status code upon detecting a match. The && symbols indicate that the echo command will be run to print out the file name with content matching the grep-ed for String if (and only when) the grep command returns a successful status (0).

All of the above are only executed against files with .jar extension thanks to the find . -iname '*.jar' command. The whole thing is piped to a shell.

Thanks to jan61 for the elegant Linux command covered in this post that makes it easy to search contents of files in JARs for a given string.

Significant Software Development Developments of 2012

I have written before (2007, 2008, 2009, 2010, 2011) on my biased perspective of the most significant developments in software development for that year. This post is the 2012 version with all my biases and skewed perspectives freely admitted.

10. Groovy 2.0

Groovy 2.0 have been an important version for Groovy. Groovy 2's arguably most notable new features are its static type checking and static compilation capabilities.

9. Perl Turns 25

Perl celebrated its 25th anniversary in 2012. Love it or loathe it, Perl has definitely become the predominant scripting language, especially in the non-Windows environments. Perl is not my favorite (I'd rather use Groovy, Python, or Ruby), but I find myself needing to use it at times, usually because I'm modifying or using an existing script or set of scripts already written in Perl. Happy Birthday, Perl!

8. Git and GitHub

Git is the trendy choice now in version control and GitHub is equally trendy for hosting code. The post Why Would Anyone use Git in their Enterprise? states, "Git has a cult-like following in the development community." The book Pro Git (2009) is freely available for reading online and can be downloaded as a PDF, mobi, or ePub electronic book.

7. NoSQL and Polyglot Persistence

The NoSQL concept seems to be maturing and moving from unabated hype and hyperbole to understanding when it works well and when it doesn't. In 7 hard truths about the NoSQL revolution, Peter Wayner writes: NoSQL systems are far from a perfect fit and often rub the wrong way. The smartest developers knew this from the beginning. ... the smart NoSQL developers simply noted that NoSQL stood for "Not Only SQL." If the masses misinterpreted the acronym, that was their problem.

Martin Fowler's nosql page states: "The rise of NoSQL databases marks the end of the era of relational database dominance. But NoSQL databases will not become the new dominators. Relational will still be popular, and used in the majority of situations. They, however, will no longer be the automatic choice." With this, Fowler introduced the concept of polyglot persistence (which he mentions was previously coined by Scott Leberknight in 2008) and explicitly compared this to the concept of polyglot programming. If we as a software development community believe that the advantages of using multiple languages in the same application (polyglot programming) are worth the costs, then it follows that we might also determine that the advantages of using multiple data stores within the same application (polyglot persistence) are also worth the costs of doing so.

6. Mobile Development

Mobile development continues to rapidly rise in 2012. The December 2012 write-up on the Tiobe Index states that Objective-C is likely to be the language of the year again in 2012 due to its rapid rise (third in December 2012 behind only C and Java and ahead of C++ and C#). The writers of that summary conclude about language ratings on this index, "In fact it seems that if you are not in the mobile phone market you are losing ground."

Suzanne Kattau's post Mobile development in the year 2012 succinctly summarizes the changes in popular mobile device platforms and operating systems in 2012.

5. Scala (and Typesafe Stack 2.0 with Play and Akka)

I have highlighted Scala multiple times in these year-end review posts, but this is my highest rating of Scala because Scala has seen a tremendous year in 2012. On 23 August 2012, Cameron McKenzie asked, "Is Scala the new Spring framework?" An answer to that question might be implied by the 1 October 2012 announcement that Spring founder Rod Johnson had joined Typesafe's Board of Directors (Johnson left SpringSource in July). Scala provides the intersection of again-trendy functional programming with widely popular and proven object-oriented programming and is part of the increasingly popular movement to languages other than Java on the JVM. It's not difficult to see why it had a big year in 2012.

The Typesafe Blog features a post called Why Learn Scala in 2013? that begins with the statement, "2012 was a big year for the Scala programming language - with monumental releases, adoption by major enterprises and social sites alike and a Scala Days conference that knocked the socks off years past." The post then goes on to list reasons to learn Scala in 2013 with liberal references to other recent positive posts regarding Scala.

Ted Neward has predicted that in 2013 "Typesafe (and their Scala/Akka/Play! stack) will begin to make some serious inroads into the enterprise space, and start to give Groovy/Grails a serious run for their money." I am not calling Play and Akka out in this post as separate significant developments in 2012, but instead lump them together with Scala as part of justifying Scala taking the #5 spot for 2012. There is no question, however, that 2012 was a big year for Akka and Play. The year 2012 saw the release of Typesafe Stack 2.0 along with Play 2.0 and Akka 2.0.

4. Big Data

Big Data was big in 2012. AOL Government has named Big Data its Best of 2012 for the technology category. Geoff Nunberg argues that "'Big Data' Should Be The Word Of The Year." Interest in the statistical computing language R has (not surprisingly) risen along with the surging interest in Big Data.

3. HTML5

2012 was another big year for HTML5. Although HTML5 continued to be evangelized as a standards-friendly favorite of developers, some hard truths (such as performance versus native code) about the current state of HTML5 also became more readily obvious.

That being stated, I think HTML5 still has a very positive future ahead of it. Although it has certainly been over-hyped with emphasis on what it might one day become rather than what it is today, it would also be foolhardy to ignore it or underestimate its usefulness. Two articles that remind us of this are FT exec: HTML5 is not dead and HTML5 myth busting. The article 'HTML5 is ready' say creators of mobile HTML5 Facebook clone talks about attempts to prove HTML5 is ready today from a performance standpoint.

2. Security

Awareness of security holes, risks, and vulnerabilities has been increasing for the past several years largely due to high-profile incidents of lost sensitive data and new legal requirements. However, 2012 seemed to be a bigger year than most in terms of increasing awareness of security requirements and expectations in software architecture and design.

Java seemed to be particularly hard hit by bad security news in 2012. Articles and posts that provide examples of this include 1 Billion computers at risk from Java exploit, Oracle's Java Security Woes Mount As Researchers Spot A Bug In Its Critical Bug Fix, Java Vulnerability Affects 1 Billion Plug-ins, Another Week, Another Java Security Issue Found, Oracle and Apple Struggle to Deal with Java Security Issues, and Java still has a crucial role to play—despite security risks.

The article Oracle to stop patching Java 6 in February 2013 suggests that users of Java should upgrade to Java 7 before February 2013 when Oracle will supply the last publicly available security patch to Java SE 6 outside of an Oracle support plan. Another article is called Oracle's Java security update lacking, experts say.

1. Cloud Computing

It seemed like everybody wanted a cloud in 2012 even if they didn't really need one. Archana Venkatraman put it this way, "2012 was the year cloud computing hit the mainstream." Steve Cerocke stated, "2012 will go down as the year of cloud computing." Other articles and posts on the biggest cloud stories of 2012 include The 10 Biggest Cloud Stories Of 2012 and Top five cloud computing news stories in 2012.

Cloud Computing is in the sweet spot many trendy technologies and approaches go through when enthusiasm is much greater than negativism. Charles Babcock's Cloud Computing: Best And Worst News Of 2012 is more interesting to me than many cloud-focused publications because it highlights the good and the bad of cloud computing in 2012.

Honorable Mention

I couldn't fit everything that interested me about software development in 2012 into the Top Ten. Here are some others that barely missed my cut.

C

As mentioned earlier, the C programming language appears headed for #1 on the Tiobe Index for 2012. One of programming's most senior languages is also one of its most popular. When one considers that numerous other languages are themselves built on C and when one considers that many languages strive for C-like syntax, the power and influence of C is better appreciated. C's popularity has remained strong for years and 2012 was another very strong year for C.

Another piece of evidence arguing C's case is the late 2012 O'Reilly publication of Ben Klemens's book 21st Century C: C Tips from the New School. The author discusses this book and C today in the O'Reilly post New school C.

Although I have not written C code for several years now, I've always had a fondness for the language. It was the language I used predominately in college (with Pascal and C++ being the other languages I used to a less degree) and I wrote the code for my electrical engineering capstone project in C. I remember (now fondly) spending almost an entire Saturday on one of my first C assignments fighting a bug to only realize that it was not working because I was using the assignment operator (=) rather than the equality operator (==). This lesson served me well as I learned other languages in terms of both what it important to differentiate and in terms of how to better debug programs even when a debugger is not readily available. I think my C experience grounded me well for later professional development with C++ and Java.

Gradle 1.x

Using an expressive programming language rather than XML or archaic make syntax to build software seems like an obviously beneficial thing to do. However, make, Ant, and Maven have continued to dominate in this area, but Groovy-based Gradle shows early signs of providing the alternative we've all been looking for. Gradle still has a long way to go in terms of acceptance and popularity and there are many other build systems with some of Gradle's ideals that have failed, but Gradle did seem to capture significant attention in 2012 and can hopefully build upon that in future years. Gradle 1.0 was formally released in June 2012 and Gradle 1.3 was released in November 2012.

DevOps

Among others, Scott Ambler predicted that "DevOps will become the new IT buzzword" in 2012. If it is not "the" buzzword of 2012, it is not for a lack of trying on the DevOps evangelists' part. The DevOps movement continued to gain momentum in 2012. The DZone DevOps Zone sees one or more posts on the subject added each day. The only reason this did not make it into my Top Ten is that I still don't see "Typical Everyday Coder" talking about it when I am away from the blogosphere talking to in-the-trenches developers.

Amber's concluding paragraph begins with this prediction, "Granted, there’s likely going to be a lot of talk and little action within most organizations due to the actual scope of DevOps, but a few years from now, we’ll look back on 2012 as the year when DevOps really started to take off." Only time will tell. There continue to be posts trying to explain what exactly DevOps is.

Departures of Noteworthy Development Personnel

There were some separations of key developers from their long-time organizations in 2012. As mentioned previously, Spring Framework founder Rod Johnson left VMWare/SpringSource (and ultimately ended up on the Board of Directors for Scala company Typesafe). Josh Bloch, perhaps still best known for his work at Sun on the JDK and for writing Effective Java, left Google in 2012 after working there for eight years.

Resurgence of Widely Popular but Aged Java Frameworks

Two very popular long-in-the-tooth Java-based frameworks saw a resurgence in 2012. Tomek Kaczanowski recently posted JUnit Strikes Back, in which he cites several pieces of evidence indicating a resurgence in JUnit, arguably the most commonly used testing framework in Java (and, in many ways, the inspiration for numerous other nUnit-based unit testing frameworks). Christian Grobmeier's recent post The new log4j 2.0 talks about many benefits of Log4j2 and how it can be used with more recently popular logging frameworks such as SLF4J and even Apache Commons Logging.

Electronic Books (E-books)

Electronic books (ebooks) are becoming widely popular in general, but also specifically within software development books. This is not surprising because e-books provide many general benefits, but also have benefits particular to software development. In particular, it is nice to be able to electronically search for terms (overcoming the poor index problem common to many printed programming books). Other advantages include the ability to have the book on laptops, mobile devices, e-readers, and other portable devices. This not only makes a particular book readily available, but makes it easy to carry many books on many different technical subjects with one on travel. It is also less likely for the electronic book to be "borrowed" unknowingly by others or turn up missing.

Perhaps the biggest advantage of electronic books is cost. It is fairly well known that technical books are generally not big profit makers for publishers. However, with printing and distribution costs being a significant portion of traditional publication costs, e-books make it easier to publishers to price these books at a lower cost than the printed equivalent.

The reduced cost to the publisher for an electronic book can be passed onto the consumer. I recently took advantage of an offer from Packt Publishing to purchase a total of eight of their books as electronic books for a total price of $40. Given that a single printed programming book can cost $40 or more, this was a bargain. I have also blogged on other good deals on e-books provided by other technical publishers such as O'Reilly and Manning.

I have especially appreciated the Manning Early Access Program (MEAP). This program is only viable thanks to electronic books and allows readers to read a book as it is developed. Because technologies change so quickly, it is often helpful to get access to even portions of these books as quickly as possible.

Finally, another advantage of e-books is their ultimate disposal. In reality, they take up such a small portion of even an old-fashioned CD or DVD, that I can usually dig up a copy if I want to. However, I can remove them from my electronic devices when I no longer need them and need the space. There are no environmental or logistic concerns about their disposal. This is important because these books do tend to get outdated quickly and sometimes an outdated programming book is worse than having no book at all because it can be very misleading.

PhoneGap / Cordova

Given the popularity of mobile development and HTML5 in 2012, it's not surprising that PhoneGap and Cordova had big years in 2011/2012. In the web versus native debate, one of the advantages of web is the portability of web apps across multiple devices. The PhoneGap/Cordova approach brings some of this benefit for writing code but maintains some of the performance advantages of running native applications.

Objective-C

Objective-C looks to win the Tiobe Index language of the year again in 2012. This is yet another indicator of mobile development prevalence as Objective-C's popularity is almost exclusively tied to iPhone/iPad development, though Objective-C's history is closely coupled with the NeXT workstations and even has been called an inspiration for Java (advertised as quoted by Patrick Naughton) instead of C++.

Kotlin

For several years now, it has been trendy for the "cool kids" to post feedback messages on articles or blogs about Java features proclaiming that Groovy or Scala does anything covered in that blog post or article better than Java does it. Many of the "cool kids" (or maybe different "cool kids" with the same modus operandi) now seem to be doing the same on Scala blog posts and articles, advocating the advantages of Kotlin over Scala.

As Scala and Groovy still lag Java in terms of overall popularity and usage, Kotlin lags both Groovy and Scala in terms of adoption at this point. However, there are definitely some characteristics of Kotlin in its favor. First, the Kotlin web page describes the language as, "a statically typed programming language that compiles to JVM byte codes and JavaScript." I could definitely see how "statically typed" and "compiles ... to JavaScript" would be endearing to developers who must write JavaScript but prefer static compilation. Andrey Breslav, Kotlin Project Lead, believes that static languages compiling to "typed JavaScript" will be a major development of 2013 and he cites Dart and TypeScript as other examples of this. Being able to run on the JVM can also be an advantage, though this is no different than Groovy or Scala.

One major positive for Kotlin is its sponsor: JetBrains. It is likely that their IDE, IntelliJ IDEA, will provide elegant support for the Kotlin language. This is also a sponsor/owner with the resources (monetary and people) to improve chances for success. Because JetBrains is "planning to roll out a beta of Kotlin and start using it extensively in production," they are more likely to continue investing resources in this new language.

There was no way I could justify to myself putting Kotlin in my top ten for 2012, but once it is released for production use, it is possible that Kotlin may make another year's Top Ten list if it is widely adopted.

Ceylon

Kotlin isn't the only up-and-coming static language for the JVM; Ceylon is also relatively young in this space. I wrote about the JavaOne 2012 presentation Introduction to Ceylon and that post provides brief overview and description information.

The first milestone of Ceylon IDE (Eclipse plug-in) was released in early 2012 and was followed in March with the release of Ceylon M2 ("Minitel"). This was followed by the Ceylon M3 ("V2000") release in June and Ceylon M4 ("Analytical Engine") in October.

The newer JVM-friendly languages with the seeming best chances of long-term viability are those with strong sponsors: Groovy has SpringSource, Scala has TypeSafe, Kotlin has JetBrains, and Ceylon has RedHat.

End of Oracle/Google Android Lawsuit

The lawsuit between Oracle and Google over Android seems to have, for the most part, concluded in 2012. There still seems to be bad blood between the two companies, but the settlement of this probably allows for continued success of the Android platform and potentially for collaboration at some future point between the two companies on Java. It will be interesting to see if Google allows its employees to submit abstracts to JavaOne 2013.

Everyone a Programmer

When HTML first started to expand across the universities and colleges in the early-to-mid 1990s, it seemed that everyone I knew was learning HTML. Most of us were "learning" it by copying other peoples' HTML source and editing it for our own use. Of course, everything then was static and fairly easy to pick up. It probably also skewed my perspective that I was majoring in electrical engineering with an emphasis in computer science and so was around people who had a tendency to adopt and use new technology. Perhaps for the first time since then, I have felt that there is an ever-growing interest in pushing everyone to learn how to program at a certain level. I don't need to provide any supporting points for this because I can instead reference Joab Jackson's 2012: The year that coding became de rigueur. Not only does this post enumerate several examples of the debate about whether everyone should learn programming, but it also makes cool use of "de rigueur" in its title.

Java

I did not include Java itself in my Top Ten in 2012. Perhaps this is an indication that I too felt that 2012 was a slow year for Java (and agree that this is not necessarily a bad thing). That being stated, Martijn Verburg has listed some "personal highlights of the year" in the world of Java in 2012 in What will 2013 bring? Developers place their bets. These include the JVM's entry into the cloud, Java/JVM community growth, OpenJDK, Java EE 7, and Mechanical Sympathy.

It's a small thing in many ways, but I think James Gosling returning to JavaOne and throwing out t-shirts was symbolic of a strengthening resurgence among an already strong Java development community.

Jelastic

Java on the cloud in general had a big year in 2012. Jelastic had a particularly big year. The screen snapshot below is from an e-mail sent out by Jelastic COO Dmitry Sotnikov.

Jelastic was prominently mentioned at JavaOne 2012 multiple times. Some of these mentions were due to winning the Duke's Choice Award for 2012. Other mentions resulted from James Gosling's positive review of his use of Jelastic. As I blogged on earlier, Gosling described himself as "a real Jelastic fan" at the JavaOne 2012 Community Keynote.

Linux-based ARM Devices

Oracle announced release of a Linux ARM JDK in 2012. The ability to run even JavaFX on Linux ARM devices provides evidence of Oracle's interest in supporting Linux ARM devices. Given that Oracle is well-known for investing in areas where returns are often great, it follows that perhaps Oracle sees great potential for the Linux ARM devices in the future. An interesting article that looks into this is Java 8 on ARM: Oracle's new shot against Android?

One couldn't go to a keynote presentation at JavaOne 2012 without hearing about one very trendy Linux ARM Device, the Rasperry Pi. Similarly, the BeagleBoard and PandaBoard have also become very popular.

Improving Job Market for Software Developers

2012 seemed to be a good year for those with careers in software development and this seems likely to continue. CNN Money ranked software developer as #9 in its Best Jobs in America story (software architect was #3). For example, Lauren Hepler has written that Software developers top 2013 job projection and cites a Forbes slides-based presentation.

Perhaps more importantly than these stories are my anecdotal observations of a surging market for software developers. I have seen an uptake in number of unsolicited employment queries from potential employers and clients. I am also seeing an increase in billboard advertising for developers, especially in areas of the United States with high concentrations of software development. This improving job market might be one of many reasons for increasing interest in programming in general.

Other Resources

There are other posts of potential interest. Katherine Slattery's Takeaways from the Top Development News Stories of 2012 talks about the Node.js "hype cycle," open source hardware, native apps versus HTML5 apps, and the "'learn to code' craze."

Ted Neward's annual predictions (for 2013 in this case) and review of his prior year (2012 in this) predictions is an interesting read.

Conclusion

2012 was another big year in software development across many different areas and types of development. Many of the themes discussed in this post overlap and are somehow associated with mobile development, cloud computing, and greater availability of data.

Late February 2012 Software Development Links of Interest

This post summarizes and provides links to some online resources related to software development that have recently captured my attention. Topics include Linux, DevOps, dynamic typing versus static typing, abstraction versus simplicity, and cloud computing.

Linux

Several posts on Linux have recently captured my interest. The post 10 free Linux e-books provides a list of ten freely available Linux books. Each book is featured with an image of its cover along with a brief description and a link to the electronic version of the book. Titles include Advanced Linux Programming (2001), Java Application Development on Linux (2005), and Linux Network Administrator's Guide, 2nd Edition (2000). One of the free referenced electronic Linux titles is The Linux Command Line: A Complete Introduction, a book that receives rave reviews from Peter N. M. Hansteen in yesterday's post The Linux Command Line Is A Very Appealing Story. The Linux Command Line has tremendous breadth, covering topics ranging from use of vi, to shell scripting to basic Linux commands. Speaking of Linux commands, the post Linux Command Line Tips that Every Linux User Should Know provides an interesting summary of Linux command-line commands.

DevOps

I remain uncommitted to the long-term value of the DevOps movement, but have certainly not made up my mind for good yet and am waiting it out to see if more substance comes from it. For example, even though I've tangentially touched on DevOps before, I still have not made a "Label" for it to mark my blog posts that cover the topic. Edward Capriolo's post DevOps has no chance is not on the fence about the subject. The post concludes with a point that I think's worth considering: "It is a left brain right brain thing. The qualities that make a good dev usually make a terrible ops person and vice versa. The world is ok that way."

Dynamic Typing

As software developers, we cannot seem to help ourselves and must continue the debate about whether static or dynamic typing is best. I've found advantages to both (that most realistic and fair developers will acknowledge), so it truly is a case of what best fits the particular task at hand as well as the task's surrounding development environment. As a general rule of thumb, I prefer dynamic typing for small efforts, especially scripting, but begin favoring static typing as a program gets larger and more people are involved. It sounds bad, but in essence I prefer static typing when I cannot trust "the other person" (which may be myself on a large enough code base). In Why I don’t like Dynamic Typing, John Mount states, "I find the pain of having to type or read through extra declarations is small (especially if you know how to copy-paste or use a modern IDE). And certainly much smaller than the pain of the dynamic language driven anti-patterns of: lurking bugs, harder debugging and more difficult maintenance. ... Initial coding is not the only phase of the software lifecycle." Mount appropriately recognizes that "there is, of course, no prior reason anybody should immediately care if I do or do not like dynamic typing," but offers some valid arguments for static typing (just as Martin Fowler has offered for dynamic typing). The reddit/programming comments/responses are a reminder that if you want a blog post to get attention, write about static typing versus dynamic typing.

Too Much Abstraction

Several experienced colleagues and I have been discussion off and on over recent months the seeming over-abstraction we've seen in certain frameworks and in code bases we've been exposed to. In the post Peak Abstraction, Tom Hammersley hits on this topic. Hammersley has done a nice job of articulating how this is especially likely to happen with a developer who is no longer new, but has not yet learned some hard lessons from more experience: "There is a common pattern. It typically occurs after 3-4 years of professional practice. A programmer is growing in his confidence. He begins to tackle more complex, challenging problems. And, he reflects those problems with complex, challenging data structures. He has not yet learnt the fine art of simplicity." Simplicity is often recognized as a desirable virtue in software development, but it is all too easy to become enamored with excessive abstraction, especially when one sees abstraction replied layer upon layer frequently in so many other aspects of software.

Anticipating Cloud Failures

In my summary of 2012 software development developments, I talked about the continuing prevalence of cloud computing in software development literature and also highlighted cloud failures in 2011 in the "honorable mention" list. The post New Year, New Security Breach; Three Potential Cloud Provider ‘Screw Ups’ to Watch cites 3 big screw-ups you can expect from cloud providers in 2012, which lists three expected cloud provider blunders in 2012: security, migration cost, and performance. The "New Year, New Security Breach" post also provides an interesting "Security Breach Timeline" for 2011.

Conclusion

There is a lot going on in the world of software development and the posts referenced above provide a very small taste of what's going on out there in the blogosphere.

Groovy, Java, and the Unix Epoch Time

The link What Every Programmer Should Know about Time was recently posted on DZone and was a highly popular link. It references the original Emil Mikulic post Time and What Programmers Should Know About It. This post and its popularity was reminder to me of how often I find myself dealing with dates and times in software development and especially when writing scripts. I like many of Mikulic's recommendations because I feel that conventions and standards can keep the intricacies of working with time zones, Daylight Saving Time, and leap seconds/leap years somewhat manageable.

I have several times appreciated the fact that Java (and by extension, Groovy) base time against the same epoch as Unix. Java and Unix both use midnight, January 1, 1970 UTC/GMT as their epoch time. The Javadoc documentation for both of Java's standard "Date" classes (java.sql.Date and java.util.Date) talk about the significance of that date. The java.sql.Date documentation states, "A milliseconds value represents the number of milliseconds that have passed since January 1, 1970 00:00:00.000 GMT" while java.util.Date documentation on the constructor accepting a long states, "Allocates a Date object and initializes it to represent the specified number of milliseconds since the standard base time known as "the epoch", namely January 1, 1970, 00:00:00 GMT." The java.util.Date documentation also goes into a discussion of the subtle differences between GMT/UTC/UT.

Because Java (and Groovy) base their date/time APIs (Date, Date, and Calendar) off of the same epoch starting time as Unix, it is easier to deal with dates and times when writing Groovy scripts in a Linux environment. The most important thing to keep in mind when converting between the two is that Unix APIs tend to use seconds since the epoch time while Java's APIs tend to use milliseconds since that same epoch time. In short, the typical conversion is to multiply Unix's seconds since epoch by 1000 to get Java's milliseconds since epoch or divide Java's milliseconds since epoch by 1000 to get Unix's seconds since epoch.

I now show a couple of simple examples. It is easy to get a Unix example. The next screen snapshot shows the number of seconds since Unix epoch time for that last time that page was updated.

As the image above indicates, the integer 1310724379 represents "2011-07-15 10:06:19Z." This is also shown with the online Epoch Converter as shown in the next screen snapshot.

It's really easy to calculate the same time in Groovy from that integer. This is shown in the next screen snapshot.

The code for the above was so simple that I was able to invoke it simply via the command line Groovy launcher with the -e option. The inline Groovy code was fairly simple and is shown again here:

println new Date((1310724379 as long)*1000)

There were some minor things that needed to be done to convert the number used for Unix seconds since epoch to Java/Groovy's milliseconds since epoch. First, there was the obvious need to multiply the Unix number by 1000 to get from seconds to milliseconds. The Groovy coercion to a long is also necessary so that the numeral is not treated as an int, leading to the wrong time. The effect of this is shown in the next screen snapshot where the numeral by itself is shown leading to the wrong time and using the more traditional Java approach of coercing the numeral to a long with the "L designation. As the image indicates, coercion to a long is vital to perform the conversion correctly.

The Groovy example shows MST time rather than GMT. There are ways to address this in Java such as using DateFormat.setTimeZone(TimeZone) or Calendar.setTimeZone(TimeZone). One can use one of these approaches to get a UTC/GMT representation of the time.

In Java or Groovy development, it is easy to save dates and times in the database as such, but it can also be effective to simply store the integer representing the time since epoch in the database. When this approach is used, my preference is to always store UTC/GMT times. This avoids the need to store an offset to have the correct time unless the offset is desired to display the time in a local timezone. Understanding the Unix Epoch (in Java), Time Zones and UTC is a good post on Java's handling of dates/times in UTC format. That post references the post Current time and System.currentTimeMillis() as a good source of information on how date/time storage works in Java. Making Sense of Java's Dates is another good source of information on Java and dates.

It is easy to determine what Java and Groovy use as their epoch time without going to the documentation. The following Groovy script does just that.

displayZeroEpochTime.groovy

#!/usr/bin/env groovy
def zeroEpoch = Calendar.getInstance(TimeZone.getTimeZone('GMT'))
zeroEpoch.setTimeInMillis(0)
println zeroEpoch.format("dd-MMM-yyyy HH:mm:ss zzz")

The output of it is shown in the next screen snapshot and verifies that 0 milliseconds since the epoch time means January 1, 1970, 00:00:00 GMT.

If Java/Groovy and Unix had different epoch times, one could calculate the difference between their starting points and continually apply that difference in conversions between the two systems. However, by using the same epoch time as Unix, Java's original creators have made it easier for those of us writing Groovy scripts that deal with time to integrate with the Unix time handling functions. That definitely makes scripting in a Linux environment easier.

Viewing a JAR's Manifest File

I like the blog post Viewing the MANIFEST.MF from a jar in a single command because it covers a common development environment need many Java developers will run into from time to time and because it covers a nice tip for handling this common need. The post looks specifically at how to use Linux unzip command with its -p option to display the contents of a JAR file's MANIFEST.MF file to sdtout in a single command and without any new files that need to be cleaned up.


Linux unzip -p

The post "Viewing the MANIFEST.MF from a jar in a single command" provides an example of how to use the Linux unzip command and its -p option. I adjust that example slightly for my own example here that I will use in demonstrating several ways to access the contents of the MANIFEST.MF file. For my example, I will be looking at the MANIFEST.MF file contained in the jdiff.jar I have built locally on my system. Although JDiff comes with a pre-built JAR, I have been building and using it locally with Java 7 and thought it provided as good as any JAR upon which to try viewing the manifest file. This JAR's manifest file can be viewed in Linux with the following unzip command:

unzip -p jdiff.jar META-INF/MANIFEST.MF

Because the -p option specifies that the unzip command should write its output to standard output rather than to an actual file, there is no need to clean up any files.


Brute Force jar Approach

The author of the blog post "Viewing the MANIFEST.MF from a jar in a single command" mentions that the "old way" to view a manifest file was often to create a temporary directory, copy the JAR of interest into that temporary directory, change directory to that temporary directory, unzip the JAR file's contents, view the file of interest, and then remove the temporary directory. That post demonstrates the commands that would be done for this approach.


Slightly Less Brute Force jar Approach

Fortunately, the jar command provides a few places to make things a little easier. Specifically, jar allows us to explicitly specify a subset of files we want extracted. This means that instead of making a temporary directory to hold all the extra stuff that we don't want to clutter our current directory, we could simple extract the single file with the jar command and this is often acceptable if it's only one file. This is shown in the next screen snapshot.

The steps shown above aren't too onerous. The single file is extracted with the command

jar xvf jdiff.jar META-INF/MANIFEST.MF

and then all that's left to be done is to run a command like (in DOS)

type META-INF\MANIFEST.MF

(cat or less or more could be used in Linux). The only residue left from this is a META-INF directory with the desired MANIFEST.MF file inside of it, but there is still something left to be removed.


Firefox

The Firefox web browser can render more than simple HTML and text files as is demonstrated in the post View Contents of ZIP/JAR File using Firefox. That post demonstrates a simple example using Firefox 3, but this feature is also available in Firefox 4 as shown in the next three screen snapshots that show opening of the JAR, drilling down into the META-INF directory and then displaying the contents of the MANIFEST.MF file.

The screen snapshots shown above demonstrate how easy it is to see the contents of the JAR in the Firefox web browser! The only "special" treatment of the URL required was to specify jar:file// at the beginning of the URL and to end it with a exclamation point. In my case, it was:

jar:file:///C:/jdiff-1.1.1/jdiff.jar!/

and the more general syntax is:

jar:file://<path_to_jar_file>!

NetBeans IDE

The Java IDEs are obvious choices for easily seeing the contents of a manifest file. The following screen snapshot demonstrates viewing the manifest file. This was easily brought up in NetBeans by simply using File --> Open File and selecting jdiff.jar to have it displayed as shown in the next screen snapshot.

JDeveloper IDE

Other Java IDEs provide similarly easy-to-use viewing of manifest files within a JAR. The next snapshot demonstrates this for JDeveloper, which allows one to see a JAR file's contents by selecting File --> Open to open the JAR file.

Eclipse IDE

If a JAR is on an Eclipse project's build path, it can be opened up to see the JAR's contents as shown in the next screen snapshot.

Groovy

When it comes to the tools for Java development, Groovy is never far from my mind as a source for building useful tools. The next code listing shows a simple Groovy script that makes use of Java's JAR-handling classes JarFile, Manifest, and Attributes to print the contents of a given JAR's manifest file to standard output.

displayManifest.groovy

#!/usr/bin/env groovy

/**
 * displayManifest.groovy
 *
 * displayManifest <<path_to_jar_file_with_desired_manifest>>
 *
 * This script displays the manifest file of the specified JAR file.
 *
 * This script is written to be very simplistic. It doesn't check to ensure that
 * the provided file is a JAR or ZIP file and, in fact, doesn't even check that
 * a file is passed on the command-line.
 */
new java.util.jar.JarFile(args[0]).manifest.mainAttributes.entrySet().each
{
   println "${it.key}: ${it.value}"
}

Most of the above Groovy script is comments. When the comments are removed, it boils down to a very small piece of code. It's true that it lacks any type of usage enforcement to ensure that a JAR file is provided, but it does the job as long as one provides a valid JAR file as the first argument to the script.


Conclusion

This post has demonstrated several different approaches to viewing a JAR's manifest file. Each approach has different advantages and disadvantages. A Java developer's IDE is conveniently available and often can be just what the developer desires. However, if the developer wishes to view the contents of a manifest file outside of the IDE, the are operating-system specific tools like unzip -p that can be used, jar can always be used directly even if brute force is required, and Groovy can be used to write a very simple script that takes advantage of Java's java.util.jar package. Another ubiquitous tool that can be used to view JAR file contents is the Firefox web browser.

CSS 2008 Linux Presentations Released

Three Linux-related presentations from Colorado Software Summit 2008 have been made available on the Colorado Software Summit site. Two of the presentations are from Thomas Cameron and are called "Automating Your Linux Infrastructure" and "SELinux for Mere Mortals (or, 'Don't Turn It Off!')." The third Linux-related presentation from CSS 2008 that was just posted is Noel Bergman's "Laptop Linux: Arrive Dependent on MS-Windows, Leave Living with Linux." Noel's other presentation, which is also available now, is called "Business Processes: Big Business' Secret Weapon Is Now Available to Everyone!" and covers WS-BPEL.

The three Linux-related presentations can be downloaded from the 2008 Agenda by Speaker Name.