Enter the OSGi – Cool shit is on the way

holistic-net has a bunch of little projects/products, all of which has something to do with IBM Connections – a sort of business social website for either intra- or internet.  Greenhouse is a good example of it.  In order to bunch our products and services which do something with Connections, we are looking into transferring those products/projects into a completely modular java project.  The framework of choice — Eclipse 4.

I am new to eclipse and OSGi development, even though I started to dabble in extending the XPage runtime with OSGi, I am not ashamed to admit that I was not sure what I was doing… Those of you on stack overflow more than likely know that I have asked many questions regarding OSGi dev and I have not been overly successful.  I am ashamed to admit that I did not first look into how eclipse development really works and decided instead to jump off the deep end and somehow expect it all to work.  Lesson learned.

Come to think of it, I learned a great deal in the past two weeks.  So much so that I am about to burst.  Moreover, today is one of those days that shows why I want to be a programmer, why I love my job, and why I cannot imagine doing anything else.  In the past two weeks I was frustrated with example projects, fearful that I will not be able to learn the concepts and get things going, and perhaps a bit down on myself.  Today was the pivotal moment when it all went click and I can say that I am not only happy with what I have accomplished, but I am also seeing the full power of the eclipse framework.  Even more than that, I think I can take what I learned with plug in management and organization and move it into all of my future project development.  (Although that is more than likely just personal preference stuff anyway)

Ok, so none of this is making much sense to “normal” people so I guess I better explain myself.

1. Separation of UI development and data management.
What I mean by this is that all you need is a data model.  This model is best in an Interface.  Aside from this, you can have a service model interface.  These two interfaces are then used to build the UI.  Actually, what I have been doing is using the model interface to build a jface bean.  There seems to be a bit more to it than building a Bean for XPages, but not too much more.  In fact, it is pretty easy when it comes down to it.  Just build an abstract bean model and then just extend it and add a bit more code into your setX methods.  This allows the GUI to not care at all where and how it is getting its data.  It only wants it to follow the interface that is located in the model plug-in.  This allows for easier GUI testing, and the gui does not have to be updated if the Data Services change.

2. Modular Design
Imagine you have two brothers.  Both of them are going to use the same budgeting software programs that you design.  Brother one, lets call David, is not too bad with computers, has no fear of it, and can follow your install directions, even if they are a bit more complex.  For him, you want to use your default design of saving information into a database.  This is the easiest way to keep track of and query your data, so why not.  You can deliver the project easily to him with the data service plug-ins necessary for JPA data persistence.  And since your data is saved in a model, you can use the service Interface to verify that you are delivering the proper functions.  Now let’s say you have a second brother named Brian.  We can admit that he is a bit of a douche and does not like fancy stuff.  He likes his toys to be slow and if he can make you run around the block a few times, he will gladly go inside, get a lawn chair and a lemonade, and he will watch you jump through hoops on that blistery summer afternoon.  You can build another service that follows the model service interface and it will handle the data the way Brian needs.  Later, when John comes along, he can decide if he wants to be a Brian or a David, you still have both service implementations and he can decide which he wants to use.

We can even take this a step further and not limit it to our annoying brothers.  We can also look at pure content.  We have an application and we want to define different roles to it.  The first role is purely user input and basic displays.  He can have an instance installed onto his computer containing the plug-ins that allow him to most efficiently perform his task.  The second role is more of an administrator.  He needs to be able to do everything that first role can do, plus this user needs to be able to perform administrative tasks and needs advanced perspectives to handle those tasks.  His instance of the application will include those plug-ins all of which extend the base UI.

3. Styling controlled through CSS
Actually, I am not sure whether or not this was an option in normal SWING UI development.  I do not think so.  Anyway it makes customizing the front end very easy.  I do not know too much about this yet, but I see it as a cool ability.

4. Event Driven application with contexts and a lot of cool shit like injection
The framework takes care of all of your application needs.  Set an object into the context and be done with it.  If the object is configured to be available, the framework will also inject it into your methods and you do not have to worry much.  (You need to be smart, but still)  This eliminates the need for a single factory class or program class that maintains critical program data. There is so much candy that I would have to write a book to get into it all.  I suggest you try it yourself.

 

This means that cool stuff is going to be done with BudgIt, or well MoneySucks as I have now started to call it.  I will be transferring everything into an Eclipse 4 application.  I am excited and cannot wait to get started.  Come to think of it, I might as well start downloading eclipse right now!

XPage dev Why Java

I recently received a comment that I would like to address:

Hey Greg. I watched the tutorial and followed it quite easily. My biggest block to Java is why should I use it? I am having difficulty understanding what the great benefits are, other than getting skilled in a new field.
I also am perplexed as to why, for example, would I need a session helper when I have session info with my standard XPage. Another thing, having an object of a Domino document and using the object to read and write fields back to my source data seems rather tedious seeing that I can read and write straight to my source data?!?
I’m missing something for sure because everyone seems to be wanting to go the Java route.
If I had 1000 users open an XPage application and view a document then why create 1000 objects, one for each document and session helpers. Does that not take needless memory?

But I am looking forward to the next tutorial!!

There are a few questions in here that I find good to ask and that can cause some debate in the professional world.  I will start off my answering my part to the question why.


Why use Java instead of JavaScript?

Some people will say, “Use Java!  Why?  Because .. white noise………”

I am sure that all of you have heard the “Because it is faster”.  Faster for whom?  The computer?  The difference is negligible.  In a test done by a colleague, iterating over a view with 75,000 documents, touching every one completed only about two seconds faster when done in Java as opposed to JavaScript (14 seconds rather than 16 seconds).  I say, if you are doing something in the real world that needs to touch that many documents and the user is waiting on you, then you are doing something wrong!  I do not care if that is done in JavaScript or Java.  Now, keep in mind that he was doing this for a test and to see if there was a measurable difference.  There was.   We thank him for doing his tests and for sharing the results with us! But to that end, lets make our first point:

Use the correct tool for the correct job!

First off let me say that I find this debate stupid.  It reminds me of the debates that I had to listen my friends engage in regarding their preferences to C# vs Java and Windows vs Linux or Mac.  Every task has its goals, and every goal has its best practice and optimized structure and technology.  If you are working on a project that will be small, used by few, and only requires a few pieces of JavaScript code, than maybe it is best to use pure JavaScript in your implementation.  Likewise, if you are going to be building an application that has code that focuses on customizing the controls, then that too might be best to be done in a pure JavaScript environment.  JavaScript simply is the tool for those jobs.  If you are going to be working on code that needs to be well documented, used by others, maintained by others, or shared between applications or one day may be shared by applications, then Java is the way to go, no doubt about it.  It just is the tool for that job!  Likewise, if you have to build code that handles jobs and working with many documents, then the code does not belong in either of them, but rather run in [scheduled] Agents.*

I know that the separation of Business Logic and UI Logic is another big push the separation of JavaScript and Java.  To that I say bullocks.  Because of the JSF infrastructure and the ability to change existing HTML generation code into actual controls, Java might indeed be the best place to store your code rather than using JavaScript to generate the code for <ComputedField>s set to HTML.  Granted customizing those controls must still be done using JavaScript, but depending on the use case, you might even have some of that logic kept in a Java Object.

Yeah, but, Why Java

Maintainability, Readability, Usability.  There is no better way to put it.

Think back to another Blog Series I did when I discussed the issues I had with debugging JavaScript.  I mentioned that the stupid error

if(apples = oranges) doSomething();

as opposed to:

if(apples == oranges) doSomething();

and that this was an error that should be caught, but never is.  This is my number one reason for pushing for java.  The stupid errors that can cost a great deal of debugging time are caught by the compiler and are easy to fix.  Another case, you have a larger function.  You are dealing with many different kinds of objects, and you accidentally try to set a document object to a documentcollection object.  In JavaScript, you will not discover this error until you run the program and you get an error that states that the desired function is not available, or you end up debugging because it manifested itself in some other way.  In java, a variable declared as a document can only be used for a document.  The error will never happen.  My first reason for pushing for Java in XPages is Usability.  You are not going to have as much code that contain errors and his hidden or uncaught by a compiler.

Another reason why I say usability is improved with java is the tooltips/inteli-sense or whatever else you want to call this feature.  If the javadoc for the API you are using is connected in the build path, you are able to look at detailed information provided by the developers, assuming that they maintained their javadoc well.  This also goes for the javacode that you write.  Of course you have the ability to write and look up comments in SSJS, but they do not show up like they do in Java.  If I am wrong about that please tell me, but I do not believe I am.  In Java, you have the ability to simply use shortcuts to import packages and classes using the inteli-sense, keyboard shortcuts, and see what members and functions are available in those classes whether they are known to designer or not.**  You write a class, that class is known by the inteli-sense. It is that simple.  It may not seem like a lot, but when you can quickly see what code does what, import, and comment all in one place, then why would you not want to?

Readability is also highly improved.  Lets say you have your classes all defined and you are using them.  In JavaScript, you do not have the ability to click on the method being called and have that method opened up in designer, at least not when those functions are defined in two different files.  With Java, this is not an issue at all.  Click on the call, file opens right to the spot in question, time is saved, finish.  The reader is just put into a position where it is efficient to go through your code and figure out what to do.  Another thing: in that same series I linked before, I mentioned that embedded JavaScript function calls are not listed in the outline view.  In Java, this is not the case.  All members, functions and embedded classes and their members are listed in the outline view thus adding to the readability of your code.

Because you are writing your code in a place that makes it readable and because it is in a very usable format that will not require a lot of learning about your code (ie reading your script files to learn how to use it and instead getting the comments via the javadoc and inteli-sense), you are saving time of future programmers who are using your code.  You are also saving yourself time when you need to change something 16 months later and do not remember what you did any longer.  You are also creating something that can be added to later!  Let me explain this.  By saying that, I am not referring to adding some code at some point down the line.  If you want to do that, than JavaScript is completely reasonable.  I am talking about finding out that there are these few classes that you use over and over and over again.  You have to port them into every database you own.  By having these classes in java already, you are in a position where it is not that expensive to port them over into OSGi Libraries that can be put on the server as globally available beans.  If you have them in JavaScript, you also need to translate them.  Furthermore, if you have an HTML generation function that you use for a computed field that displays html code, then it is not so far off to take that code one day and make a real JSF control out of it.  Likewise if you decide to make that code into a control, you can share that control via an OSGi extension.  Overkill?  Nope.  You are simply writing your application in a way that allows for future sustainability.

Why Java?  Not for the negligible speed increase, but for Maintainability, Readability, Usability.


Why a SessionHelper object?

This question refers to a video that I posted as a part of my Java XPage tutorial.  I created a Java object in order to access the session.  First off, I just would like to mention that this is primarily a class to assist other Java classes and not JavaScript.  I most likely would not put this into the faces-config file and register it as a bean, although I have done so in the past.  As the object stands right now, it is merely a list for static methods.  This is a poor candidate for a bean as an instance of the class is not necessary to use its functionality.  Get it when you need it, otherwise leave it.  The only functionality in this particular SessionHelper class that might be helpful down the line in JavaScript is the getODASession().  Otherwise, if I remember correctly, ODA creates a bean called “openSession” that allows you to access the ODA session object simply in JavaScript without any sort of self made utility class.  Even then, you could always access the ODA Factory utility.

There will be some who will say that my approach to getting a session in an external class is a bad idea.  I can only say to that that I have not had any issues getting the session or the database from other helper objects and methods.  An absolute no-go would be getting the objects under the database object.  Your cleanup would either be messy, or give you NotesExceptions depending on how you went about it.


 Java and Object Retention

Beans and the scoped maps are there for a reason.  Lets assume we have an external database that holds language information: labels, keyword lists and so on.  A typical approach from a few applications that I have taken over require that each form retrieve its own values using @DbLookup to fetch them from the source DB.  In XPages, this is dumb.  Likewise, this would be an asinine thing to do for a computed field or hiddenText field, it would be a horrible idea to put it into the viewScope, a dumb thing for sessionScope, and perfect for an applicationScope Bean which maintains a  HashMap of document labels and keywords based on some key and correctly retrieved by reading out the sessionScope bean that contains the preferred language information for the current user as read from his user document.  By putting it into the applicationScope in a bean you are creating a way to cache your expensive values and provide a way to reset those values at will.  Of course you can do similar things with JavaScript, but you cannot put a JavaScript “object” or function into the scoped maps.  We will be getting into such patterns later on in the tutorial.


Conclusion

As Scotty said in Star Trek V, “How many times do I have to tell you, the right tool for the right job!”  There are times when Java is the right tool, there are times when JavaScript is the right tool.  Do not let yourself be put into the position of thinking that only one option is correct, but keep an open mind and remember that future sustainability is directly influenced on your programming technique and tool decision.  Don’t get caught up in the stupid debate of which is better.  Do not listen to the “java is faster” crap.  The real reasons go much deeper than any of that, and if you are having performance problems, then there are much more likely causes than the language choice.

Java is also the only place where you can define classes for beans to be used with scoped maps.  Otherwise, you are just saving raw variables into your map and cluttering it up.  Use Java to correctly store and access your information so that it can be readily accessed and executed.  I am a strong supporter of Java use in XPage applications.  I hope that this article has helped explain why.

 

* This is really a case by case decision and must be decided upon specifically when designing the application.
** By “known to designer”, I am referring to basically hard coding the inteli-sense.  I actually do not know how this works and am merely guessing.  Does not matter though.

OSGi development issues

Forward

This is a post written out of frustration and the hope to get some assistance figuring out what I’m screwing up.  I did not proof-read it (will soon though), and I am certain there is some detail that is missing.  This is more of an article for me to remember what I’ve learned so far.  If you’re interested, go for it!


 

A few weeks ago, I started setting up a development environment with eclipse where I could mainly view the extension library code and perhaps edit and add to it.  This eventually was extended in the hopes of creating my own OSGi Plug-ins.  Let me quickly describe the deployment strategies that I have seen and partially used myself.

1. Central template

The first deployment strategy is going to be the clearest for Notes Developers, I believe.  In fact, it is very common that there is a central application that serves as the main design template and that all other corporate applications use that database.  Most of the time, this primary template contains very little more than images, css files, and maybe a few administrative views which assist with conflicts.  Often the next level of templates will increase the functionality to include libraries and so on.  Of course, this can be done with XPage elements and with Java classes and so on.

This is the method that I started off with, and still use in a few cases, but I found a few issues, or at least I had a few issues.  More than likely, I just found a few cool new ways of messing up and did not realize how.  My main issue was when I distributed Java source code in this manner, I ended up with a great deal of SecurityExceptions and the occasional Serializable- or ClassCastException.  The program would run for a while without any issues whatsoever, and then suddenly I would be getting these stupid errors and they would not go away.  I could only solve it by going into designer, refractoring to change the name of the offending class, and then change the name back again.  I had tried to clean the project, rebuild the project, and there were even a few times I deleted the entire class and recopied the text.  (That last one also worked)  This was annoying.  To fix this issue, the cleanest option I found was to package the source code in a jar (easy to do in eclipse) and then distribute the jar.  This can also be completed with the same sort of template distribution, but the source code is not present, or taken off the build path.  I have had problems just removing the source or old jars from the build path though.  It seems there is a cache that is present that gets in the way, so it is best just to delete it.

2. I vaguely remember some other strategy that I am not going to touch upon….

3. OSGi Development

This is the way I want to go.  The main push to develop my utility files in this fashion is my desire to publish them on openNTF.  Of course this is not any sort of requirement that I have read about, I just find it cleaner, more professional, and easier to maintain and update, not just for me, but also for the customer.  Basically, my components and java files will work in a way similarly exactly like the Extension Library content.  I can have them installed on designer and on the server and just go to town.  At least that is the theory.  There are a few obsticals that I want to touch upon here because I am still not able to do this the way I should and I am pretty stuck.  I do not know if my environment is set up incorrectly, if I am just being stupid, or what the heck is going on.  I am to the point where I do not even know what to ask anymore to try to figure out the issue.  Instead, I am just going to write about my experiance here as best I can in the hopes that someone can help me figure this out.


Eclipse Setup

Setting up eclipse in order to get this to work was my first chore and did not go well.  Again, my goal at this point was to set up eclipse in such a way that I could play with the code for the Extension Libraries.  I starting using this video to set up my environment.  I did have a few issues with this.  The main issue was that the Development package described in the video is not available.  I tried for hours to somehow find it, but it was simply sent to nothingness.  Wonderful.  Using this StackOverflow question, some help from Paul Withers, and a great deal of playing with build paths and workspace settings, I eventually got the code loaded with only a few errors.  The errors that I had were mostly in test packages from the ExtLibs.  I closed those packages and hid my head in the sand refusing to see any of the numerous warnings that I also have, and still have.  The one that bothers me the most is the “Access Restriction” warning saying that a particular class or whatever is not accessible.  I somehow managed to get them to go away (mostly) but I do not remember what I did. If I do, I will post it here.

After a few weeks of a break, I picked it back up again because I wanted to develop my own stuff.  I want to rely on the openNTF Domino API as well as the Extension Library.  Because of this, it makes the most sense to work in that same workspace and just add my projects to it.


 Noob Problems

I will confess right here and now that I am not well versed on OSGi, Plug-in development, or JSF.  I learn by doing, and if I can find a book on the subject, I will buy that mother and soak it up from cover to cover and complete every single understanding exercise it contains, or make up my own in case the author opted out of such a cool idea.  **Hint Hint to all of you authors out there that this person might buy your book if you can come up with something that deals with Domino OSGi dev**  

First off, it is not enough to just copy your source code from designer to eclipse and call it a day.  There are a few different types of projects that you have to create.

1. Project file

In my case, this Project was named de.holistic.utils.  It is the heart and soul of what I am trying to do.  It contains all of the code from designer and is what I want to publish.  I made this a plug-in project, and I am not entirely certain this was correct, but that is the current stand.

2. XSP Plug-in

This is a plug-in that I was not aware was mandatory.  In all honesty in only makes sense though.  This plug-in will point to the “com.ibm.xsp.Library” extension point.  This should act as a flag (as far as i can tell) that tells designer (and I bet Domino) that it is an addition to the Xpage runtime.  I am not really doing anything here other than setting up the XSPLibrary and Activator classes.  Additionally, I have a domino-faces-config.xml file which I have found out can store bean information like the faces config file in the .nsf’s.  My current stand is that I just copied the file directly from the development database and I have not cleaned it up.  The only thing I did was to make sure that the class names are all correct.

3. Feature project

The feature project seems to be fairly simple.  As far as I can tell, it is just a glorified library of plug-ins that are going to be installed into the runtime later.  I do not think there is any real magic happening, it is just an xml file with a pretty front end.

4. Update Site Project

The update site is also just a glorified XML file that points to one or more features.  This produces what we can later zip up and send to customers to install exactly like the ExtLibs.  Again, no magic.

Just getting this far seemed to be a bit of a hassle.  I had no idea that so much was involved in getting one jar file out onto the server.  It does make sense, but I was not expecting it.  Your welcome for the heads up.  😉


 Current Stand and Issues (my Noob-i-ness is painful – especially to me)

Here is the point where I say that I do not know where I am going wrong.  I have posted a few more questions onto StackOverflow mostly because of the symptoms I am seeing, and not really understanding what my issue is to begin with.  The problem that I am having is that I cannot access my classes in a project.  There is no further problem that that.  The rest is all guesswork as to what it can be caused by. Using this article and the answer to my question from SO, I discovered I needed to create an xsp plug-in and correctly set the extension point.  I have since done this.  That tutorial is mainly component based, and while I am sure this is going to come in handy in the future as some of my utilities are components currently used as custom controls.  I have not really had much luck figuring out what to do with java libraries and adding that to the developer’s toolbox.  The issue that the library cannot be chosen in the xsp.properties is still a concern, although i have set the XSPLibrary to global, so that should not be too much of an issue right now.  It is mostly something that I need to clean up in the future.  **I THINK**

When I run an Xpage right now, the beans that I defined in the configuration cannot be created.  It cannot find the class.  I have checked the spelling (but since I am in a really bad mood right now — yes bloody shirt type bad mood for those who read my last latest blog series) I am not so certain that I did not mess that up.  I have made the utility plug in project required for the Xsp project and I added it to the build path.  I have tried many different settings and many different combinations.  I would not be so surprised if I have shot myself in the foot by now and changed stuff that is the cause for my own issues.

The code has been published at GitHub under: https://github.com/GregReeder/de.holistic.utils
I am pretty new to GitHub, so you will not see too much from me there.  I also like to keep my code local and am not sure how often I will publicly commit code, but in this case, it is the best option.

As soon as I figure out what I am screwing up, I will be sure to post about it.

Thanks for the support!