Efficient Java Development for the Raspberry Pi

This Instructable describes a very efficient approach for developing Java programs for the Raspberry Pi. I've used the approach to develop Java capabilities ranging from low level device support to multi-threaded and network-based programs. The approach is not only efficient, it is free!

Fundamentally, the approach is remote development using NetBeans. NetBeans runs on a workstation (a desktop or laptop computer, not the Raspberry Pi) connected via Wifi to the Raspberry Pi. Once everything is set up properly, you can write a Java program in NetBeans and then a single click compiles and builds the program on the workstation, downloads the program to the Pi, runs the program on the Pi, and delivers output to the NetBeans console. Magic! You can even debug the running program, setting breakpoints and examining variable values. More magic!

The rest of this introduction describes the motivation for using Java and for remote development using NetBeans. If you care about motivation, read on. If you don't care, go to step 1.

Why use Java on the Raspberry Pi?

There are many programming languages available for the Raspberry Pi, but I'll limit the discussion to "professional grade" programming languages that target standalone programs, support multi-tasking, allow hardware and network access, etc. That means Java, Python, or C/C++ . The most recent releases of Raspbian include development and runtime support for all these languages.

Python is in effect "promoted" for use on the Pi, at least in part for its ease-of-learning for new programmers. I am an experienced programmer and am not fond of Python due to an admittedly subjective dislike of a number of its characteristics, for example, dynamic typing. There is one characteristic of Python for which my dislike is objective -- performance. This source shows that Java is always faster than Python 3, and in fact can run hundreds of times faster, depending on the task.

C in my opinion, is a "high level machine language", i.e., something that is very close to the system hardware, or at least the operating system. You can do pretty much anything, and you can do it about as fast as possible; in fact, this source suggests C can run up to 7X as fast as Java. I am also not fond of C since I (subjectively) dislike some of its characteristics that I find arcane, archaic, or both; for example, explicit pointers. Objectively, the fact that you can do anything means that you are only an obscure bug (e.g., bad pointer arithmetic or an errant memcpy) away from overwriting memory and potentially crashing the program or even the entire system. Java prevents such hazards, so Java is safer. I feel the safety also improves programmer productivity. I consider C++ an object-oriented "wrapper" around C that does nothing to eliminate the hazards of C.

The bottom line: Java is faster than Python. Java is safer than C.

Are there limitations when using Java?

So far, I've been able to do everything in Java that I can do in Python. There are things that one can do in C that one cannot do in Java, but again, so far everything I wanted to do has been addressed by Java fans in the Pi community. I'll offer an example in a later step. That said, I have purchased sensors from vendors that supplied "drivers" for the sensors only in Python (or C/C++ for the Arduino), so I had to write the Java equivalent; the extra effort is not necessarily productive, but can result in greater insight into a sensor's operation.

Why remote development based on NetBeans?

I've found that NetBeans-based remote development greatly increases productivity when using Java for the Raspberry Pi. I am confident that you can use the approach to create fast and safe programs, due to the inherent characteristics of Java, while writing, building and debugging them much more quickly than other approaches I've found, due to remote development using the "professional grade" NetBeans Integrated Development Environment (IDE).

It might be useful to discuss briefly those "other approaches". All meet the basic criteria that the Raspberry Pi does not have a keyboard, mouse, or display connected. There are many reasons for this criteria, but my primary reason is that I develop mobile robots, and I don't want the Pi connected to any peripherals except robotic subsystems, even during development.

The VNC server included in Raspbian provides a remote graphical user interface (GUI) that allows a form of remote development, where only the remote GUI runs on the workstation and everything else, writing, compiling and building, takes place on the Pi. With VNC it is easy to use the Java-capable development tools that ship in Raspbian, like BlueJ, Geany, and Greenfoot. Some of these can be considered an IDE; however, I find them far from professional grade.

It is technically possible to install an professional grade Java IDE like Eclipse or NetBeans on the Raspberry Pi and use it remotely via VNC. Reports, and common sense, suggest that the results would be disappointing, due to the large memory footprint and CPU power such IDEs require, as well as network latency introduced by a remote GUI.

Both Eclipse and NetBeans run very well on a workstation. With Eclipse, writing code is a pleasant experience, however, building the program is not; downloading the program must take place outside of Eclipse; executing the program on the Pi is completely independent of Eclipse. The overall result is a multi-step, almost painful, process that cannot support remote debugging. NetBeans has built-in compile, build, download, remote execution, and remote debug, making it a superior approach.

Step 1: Set Up the Raspberry Pi

To use the remote development approach you must start by setting up the target Raspberry Pi. There are many resources, including Instructables, available to assist in setting up a Raspberry Pi. See this for the basics of setting up the Pi (and more).

This Instructable assumes you

• install the latest Raspbian; make sure you record the Pi user ID and password
• set up Wifi and connect to your router (see this); make sure you record the Pi's IP address
• enable SSH (see this) to allow remote access to the Pi from the workstation

Once you've completed the setup, you can power down and disconnect the display, the keyboard, and the mouse from the Pi. This is one of the key advantages to remote development. You must leave any Wifi dongle connected (not needed, of course, for a Pi model 3 or Pi Zero W), and then you can power up the Pi.

To test, open an ssh client on your workstation, e.g., Terminal on MacOS or putty on Windows. Then enter the ssh (secure shell) command with the following form:

ssh the_pi_ip_address -l the_pi_user_id

You should get a prompt to enter your Pi user ID's password. If no prompt appears, check to make sure you have entered the right IP address in the ssh command. Enter your password (it will be hidden) and you should then see the Pi's shell prompt that appears something like this:

pi@raspberrypi:~ $

If you don't see the prompt, check the user ID and password you used.

Now you must find the path to the Java runtime and confirm the default version is Java 8 (should be true for the latest Raspbian). To find the Java runtime path, in the ssh client enter the command

sudo update-alternatives --display java

You should see a response with the first two lines that look like the following:

java - auto mode
   link best version is /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/bin/java

The "-8-" in the second line confirms that the default runtime is Java 8. Record the path in the second line as you will need it to configure NetBeans for remote development in a later step.

If the default runtime is not Java 8, in the ssh client enter the following command to switch to Java 8 (assuming it is installed)

sudo update-alternatives --config java 

Step 2: Install NetBeans on the Workstation

Now you need to install NetBeans on your workstation. Point your workstation browser to the NetBeans download page. You will see several possible bundles that support the two editions of Java as well as other languages. All you need for Java development for the Raspberry Pi is Java SE, but you can get the Java EE bundle or the All bundle. Once you decide which bundle you want click the corresponding Download button. You can find additional installation instructions for your workstation OS here.

Once you've installed NetBeans, start it (it may do so automatically after installation). You should see the NetBeans main window as shown in the picture. The picture was taken on a Mac, and the main window might look a bit different on Windows.

Once you've successfully installed and started NetBeans, proceed to the next step.

Step 3: Configure the Raspberry Pi As a Remote Platform in NetBeans

The following actions configure the Raspberry Pi as a remote Java SE platform for NetBeans. This enables NetBeans to download and execute Java programs on the Pi. You can find a somewhat generic description here.

NOTE: In this and following steps, I show values specific to my environment for various aspects of configuration and coding; yours will obviously be different.

To configure the Pi as a remote platform:

1. In the NetBeans main menu bar, select Tools -> Java Platforms. You will see the Java Platform Manager popup window (picture one).
2. Click Add Platform at the lower left. You will see the Add Java Platform [platform type] popup (picture two).
3. Choose Remote Java Standard Edition. Click Next. You will see the Add Java Platform [set up remote platform] popup (picture three shows the popup after I entered the proper values for my environment in the fields).
4. In the Platform Name field enter a name for the Pi. You can use pretty much anything, but it must be unique.
5. In the Host field enter the IP address for the Pi found in step 1.
6. In the Username field enter the user name you used in step 1.
7. Leave Use Password Authentication selected and in the Password field enter the password you created in step 1.
8. In the Remote JRE Path field you must enter most of the path to the Java runtime on the Pi. In step 1, it was /usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/jre/bin/java. However you must drop off /bin/java in the field value.
9. NetBeans downloads the runnable program to a working directory on the Pi, identified in the Working Dir field. The default is /the_Pi_user_ID/NetBeansProjects/ and it perfectly acceptable. You can use something different if you desire, though it must be available to the_Pi_user_ID (picture three).
10. Click Finish at the bottom right of the popup. After processing, you should once again see the Java Platform Manager popup window. Now you should see your remote Pi (under the name you used in action 4 above) under the category Remote Java SE (picture four).
11. Click Test Platform at the lower left to test the connection between NetBeans and your Pi. You will first see a popup saying Verifying Remote Platform. If successful, you'll get a second popup indicating the Connection ... successfully established. If not, you need to check and correct the information you entered into the Add Java Platform [set up remote platform] popup in actions 5-9 above. You can do so from the Java Platform Manager popup; simply select your Pi platform and then edit the fields on the right side of the popup.
12. After a successful connection, click Close in the Java Platform Manager popup. You will now see the NetBeans main window.

Now the real fun can start!

Step 4: Create a Java Project on the Workstation

This is not intended to be a full explanation of how to use NetBeans, or Java, but I will describe the minimal actions to write a program in NetBeans on the workstation and, eventually, run the program remotely on the Raspberry Pi.

In NetBeans you must first create a project to hold the Java class(es) for the program. Further, in Java, classes reside in packages to support organization and security. To create the project and optionally create a package and a class file:

1. In the NetBeans main window, click the new project icon (2nd from left). You will see the New Project [choose project] popup (picture one).
2. The defaults (Category: Java, Project: Java Application) are correct for this example, so simply click Next. You will see the New Java Application [name and location] popup (picture two shows values for my example). NOTE: I have redacted my user information using the colored rectangles.
3. In the Project Name field, enter a valid Java project name of your choosing. The name must start with a capital letter, and convention suggests camel case when concatenating words.
4. The Project Location field controls where the project resides in your file system. The default varies depending on operating system, but is safe to accept.
5. The Project Folder field allows you to control the folder name for the project. I find it best to use the default, which is a concatenation of the Location field and the Name field.
6. When the Create Main Class checkbox is checked NetBeans automatically creates a package and a main class file (a program that can be run from the command line) with the same name as the project. I think usually it should be unchecked, but in this case, I'll leave it checked which eliminates explicit actions otherwise needed to do so (picture two).
7. Click Finish to create the project, a package and a main class file. Now the top left pane of the NetBeans main window shows your project, which contains a single package, which in turn contains a single class file with a main() method. The top right pane contains the default source code for the main class (the program) automatically generated by NetBeans (picture three).

At this point, you could write some code in the main method and run it on the workstation. That may be desirable at times, but is unneeded for this Instructable, so proceed to the next step.

Step 5: Configure the NetBeans Project to Run Remotely on the Raspberry Pi

To configure the NetBeans project, and a program it contains, to run remotely on the Raspberry Pi, do the following:

1. Right or 2-finger click (depends on workstation OS) on the project in the Projects pane of the NetBeans main window to bring up the project menu and click Properties. You will see the Project Properties popup (the picture shows the popup with the proper values for my example).
2. Under Categories on the left, select Run.
3. Click New to the right of the Configuration field. In the resulting Create New Configuration popup, enter a name for the configuration and click OK. The name can be anything; I just reused the name of the platform ("My Pi"). Once again you'll see the Project Properties popup.
4. Click the dropdown icon on the right of Runtime Platform field. From the list in the popup, select the remote platform you created earlier (in my environment, "My Pi").
5. Click OK to set up the remote platform for the project. You will once again see the main window.

You are now ready to write code and run the program remotely.

Step 6: Write a Java Program on the Workstation and Run It on the Raspberry Pi

The right panel in the NetBeans main window shows the main class file generated by NetBeans. Type a simple print statement in main method of the class file (picture one). It is now a complete Java program that does something, though something very simple.

To run the program on the Pi, click the Run icon (the green left-pointing arrow) in the main window. NetBeans compiles the code, builds a jar file, downloads the jar file to the Raspberry Pi identified by the remote platform, executes the program on the Pi, captures the output (System.out) of the program, and echoes that output to the NetBeans Output pane at the lower right of the main window (picture two).

You've now significantly expanded your skill set for developing projects on the Raspberry Pi! But wait.... There is more!

Step 7: Debug the Java Program With NetBeans

One of the most powerful features of an industrial strength Integrated Development Environment like NetBeans is the ability to debug the program while running. NetBeans allows you to leverage the full suite of debugging tools while running remotely. This includes setting breakpoints at "important" or "troublesome" locations in the program and examining the value of variables at the current execution state.

The simple program in the last step is too simple to demonstrate debugging, so I added some additional code to the program. The additional code adds two variables.

To leverage the debug tools, you must first set one or more breakpoints. To set a breakpoint, click on the line number where you wish to suspend execution; a red rectangle icon appears, and the entire line acquires a red background (picture one).

To debug the program running on the Pi, click the Debug icon (immediately to the right of the Run icon) in the NetBeans main window. As with normal execution, NetBeans compiles, builds, downloads, executes, captures output, and echoes that output. But with debugging, NetBeans stops execution at breakpoints and allows you to examine the execution state (picture two, showing the Variables pane at lower right).

Execution stops at the breakpoint prior to executing the instruction on the line with the breakpoint. The green background indicates the next statement to be executed. Thus in my example, with a breakpoint at line 9, variable a does not yet exist, much less have a value.

Now hover the cursor over the debug menu icon in the main window (the first double-arrow-down icon to the right of the Debug icon) and click the Step Over icon (it has a blue document in the background and a green counterclockwise arrow in the foreground). [NOTE: if you make your NetBeans main window wide enough, the debug menu expands and the debug action icons show up to the right of the Debug icon.] NetBeans executes the statement at the breakpoint and stops execution prior to the next statement. You will now see that the variable a exists and has the expected value (picture three).

Click the Step Over icon twice. You will see that all three variables have values; note that c is still zero (picture four).

Click the Step Over icon once. You will see that variable c now has the proper value, i.e., the sum of variables a and b (picture five).

Finally, in the debug menu, click the Run icon (a green circle with a white arrow). That causes execution to continue to the next breakpoint, or to the end of the program, whichever comes first. Now the Output pane appears and shows the same output as with normal execution, in this case the text in the print statement.

Step 8: Execute the Program Independent of NetBeans

The time may come when you want to execute your program on the Raspberry Pi, without "assistance" or "interference" from NetBeans. It is very simple. First, open an ssh client to your Pi.

Remember from step 3 that NetBeans places your project jar in a working directory on the Pi. The default base is /home/pi/NetBeansProjects for user pi. The specific directory has the same name as your project. NetBeans places the jar in a subdirectory called dist. For my example, the full path is /home/pi/NetBeansProjects/MyRemoteProg/dist. In the ssh client on the Pi, to execute the program you can use the java command with the 'jar' option:

java -jar /home/pi/NetBeansProjects/MyRemoteProg/dist/MyRemoteProg.jar

You will see the results of the execution in the ssh client; in my example, the output will be

Hello from Raspberry Pi!

This works because information in the jar file identifies which main class in the jar to execute.

You can also change the working directory (via cd) to the location of the jar file and then issue a shorter command to produce the same results.

java -jar MyRemoteProg.jar

Step 9: Build and Download, But Not Execute the Program Via NetBeans

There are situations where you want NetBeans to build and download your program, but not execute it. One such situation occurs when your program needs keyboard input. If this is the case, using the technique in the previous step, once you build and download, the program executes on the Raspberry Pi and so is expecting keyboard input from the Pi, but of course, there is no keyboard attached to the Pi, so the program hangs -- not good.

You have to use a somewhat different technique to execute the program. Fundamentally, you create a "dummy" main class file that essentially does nothing, and identify it as the program to be executed after download. In this case, the "dummy" gets executed, and you can then run your "real" program directly on the Pi.

To use this technique, first create a new main class:

1. Bring up the project menu as in step 5 and click New->Java Main Class. You will see the New Java Main Class popup.
2. In the Class Name field enter a name. The name can be anything you like; I used "Dummy". The Package field allows you to identify the class' package; you select package using the dropdown to the right of the field. The class can be in the same package as your "real" program or in a different package; I put the new class in the same package. Click Finish. You will now see the NetBeans main window with the new file in the top right panel (picture one). I added a simple print statement about running on the Pi, but nothing is really needed.

To run "Dummy" after download you must make it the "main class to execute by default" for the project:

1. Open the Project Properties as in step 5. Select the Run category. Click Browse to the right of the Main Class field. You will see the Browse Main Classes popup (picture two).
2. Select the "Dummy" class. Click Select Main Class. You will once again see the Project Properties.
3. Click OK. You will return to the main window.

Now when you click the Run icon, NetBeans complies/builds the entire project, downloads the jar containing all the class files to the Pi, and executes the "Dummy" class (picture three).

To execute your real program on the Pi, you must now use the java command with the 'classpath' option, which requires that you explicitly identify the main class to execute. From an ssh client go to the dist directory for your project. The proper command is of the form:

java -cp project_name.jar package_name.class_name

The project_name obviously refers to the project; that changes only when dealing with a new project. The package_name of course refers to the package, and the class_name refers to the main class to execute. As said earlier, a project can contain many packages. Similarly, a package can contain many classes, and all could be main classes, or programs, that can be executed from the command line. Thus, the 'classpath' technique is quite useful for developing sophisticated programs with many classes, some of which are main classes only to assist unit testing.

The following command executes my example program:

java -cp MyRemoteProg.jar myremoteprog.MyRemoteProg

Step 10: Leverage NetBeans Library Management

One of the toughest but most important aspects of sophisticated programs is dealing with libraries, i.e., code someone else has already written. NetBeans does an admirable job of removing most of the pain of library management.

I'll use as an example Pi4J, which gives Java programs access to the Raspberry Pi GPIO, I2C bus, and other hardware I/O. Pi4J is but one example of support for doing lots of fun things on the Pi with Java.

You must first download the library to your workstation. See the Pi4J download page. Since you are not installing Pi4J on the Pi, download the zip to your workstation. You may or may not have to explicitly unzip once the zip is downloaded.

Now you must create a "global library" in NetBeans:

1. In the NetBeans main menu, click Tools ->Libraries. You will see the Ant Library Manager popup (picture one).
2. Click New Library at the bottom left. You will see the New Library popup (picture two).
3. Type in a meaningful name you want to use and click OK. You will once again see the Ant Library Manager popup. It now shows the new library you created (picture three).
4. Click Add JAR/Folder on the right. You will see the Browse JAR/Folder popup (picture four).
5. Navigate to and then select pi4j-core.jar. Then click Add JAR/Folder. You will return to the Ant Library Manager popup.
6. In the Ant Library Manager popup click OK. You will return to the NetBeans main window.

You've added the library so that it can be used in any project. Now you must add the library to your project:

1. Bring up the Project Properties popup (see step 5) and select the Libraries category (picture five).
2. Click Add Library on the right side of the popup. You will see the Add Library popup (picture six).
3. Navigate to the library and select it. Then click Add Library. You will again see the Project Properties popup. The library now appears in the list of Compile-time Libraries in the popup.
4. In the Project Properties popup click OK. You will return to the main window.

You've added the library to your project so that your program can use it. You can do the same sequence of actions for any libraries you need for a program.

There are three times you need the library -- compile, build, and execution. Fortunately, with NetBeans, adding the library as shown above handles all. To confirm this, I've created another main class that does the absolute minimum necessary to use the I2C capability in Pi4J (picture seven). The fact that there are no errors showing means that the class TestPi4J compiles. Clicking Run successfully builds and downloads; NetBeans downloads the library in addition to the jar file, so the program will execute. To verify the latter, you can use the technique in step 9 and in the ssh client enter the following command (from the dist directory):

java -cp MyRemoteProg.jar myremoteprog.TestPi4J

It is instructive to understand how NetBeans handles libraries. While in the dist directory for your project, list the contents of the directory (use the ls command) and you will see a lib subdirectory. List the contents of that directory and you will see the jar file identified in the global library, which in my example is pi4j-core.jar. All libraries that you add to the project will appear in the lib directory, and thus are available to all of the programs in the project jar file.

Step 11: Go Have Fun!

I've described an approach for achieving very efficient Java development for the Raspberry Pi. The Raspberry Pi is a hugely popular technology for a broad spectrum of projects. Java is an professional grade programming language that offers performance advantages over Python and offers safety advantages over C/C++. NetBeans is an professional grade IDE that greatly increases programmer productivity.

I think the combination is quite compelling. Now, go have fun using the combination for your projects.