Java Beans and funny stuff

I have such a backlog of posts that I have written and have neglected to post, it is not even funny.  I have a few videos that still need to be edited or redone and it is nuts. So first off, I am sorry for the long breaks in posts, but life tends to get in the way.

Today I was working with an apprentice and getting him involved with XPages.  He seems to enjoy it so far, and it is a help for me because I give some of the more mundane things to him.  He has zero Notes/Domino experience and has focused primarily on .NET development.  If you go to derultimativeligatipp.de, you can see a bit of his handy work.  Of course he worked with another apprentice of ours to build that site, and I must say together they built one awesome application.  But I digress.  Of course I think our apprentices are amazing, but we want to quickly discuss our adventures of the day with Beans and XPages.

As readers may or may not be aware, I have spent a great deal of time developing a standard java library that I use in all of my XPage applications for use both internally and for customers.  It includes custom controls, standard beans, and now a configuration.  But a simple configuration was not good enough for me.  Let me quickly get into my use case.

Up until now, I have been using a file saved under the Resources/Files tab of the project.  I wanted to get around needing profiles which can be a cached pain in the rear, I originally did not want to have a view to look up a specific document, and I did not want to touch the xsp.config xml file. Of course there are some wonderful snippets available from Paul Withers in case you would prefer that approach.  I wanted to save values that differ from database instance to database instance, as well as from dev version to QA version to productive version.  As far as I am aware, performing a design refresh also rewrites the config.xml file.  Really the best way to get the functionality I wanted was a good old fashioned NotesDocument, a good old fashioned view, and the wonderful ODA view method “getFirstDocumentByKey()”. #ODAOnceMoreForTheWin .

This brought on interesting points that I could discuss with the apprentice: abstract classes, Beans, and expression language.   I wanted to build the following structure:

AbstractConfig
-contains the view alias for lookups which will be used for all XPage configurations
-abstract init() method
-abstract getLookupKey() method
-a few other methods that are sensible for our use case but may not be needed for all.

AbstractDashboardConfiguration => AbstractConfig (just an example)
-implements init() to fetch the needed data
-protected final Strings for each field in the NotesDocument
-private members for each field value and the appropriate getter/setters

DashboardConfigurationDocument => AbstractDashboardConfiguration (just an example)
-save() implementation
-specific XPage rendering helper functions
-is set up as a viewScope variable

DashboardConfiguration => AbstractDashboardConfiguration (just an example)
-Specific methods to use a re-init interface that I built so that all of my applicationScoped beans can be reinitialized with a click of a button by an admin
-obviously applicationScoped

As you can see, the build up is pretty complex, and this is the easiest of examples.  There are probably a few “WTF?” questions that I should touch upon, so let me get to them first.

First off, I am sure the reason for an AbstractConfig class is clear.  When all configuration documents are already being saved in the same view, then why not.  Certain fields need to be set in order to guarantee that they are displayed in the correct view. Also, why should the name of the view be set in each configuration class?  It just makes sense to have a single abstractconfig java class.  But, the question that probably comes to mind is, why is there a need for another abstract class before the real implementation?

The answer is pretty simple: I hate StackOverflowExceptions.  I started to create two classes to handle configuration information.  One bean would be responsible for saving and editing the information (DashboardConfigurationDocument), and the other would be responsible for caching the information in the applicationScope (DashboardConfiguration).  Without the abstract class I am left with the following conundrum…..

It is clear that DashboardConfigurationDocument should get it’s information from the document… I mean….  it is sort of implied in the name.  It should also save itself. It then needs to inform the applicationScoped DashboardConfiguration bean that it should refresh its data. This data could be read from DashboardConfigurationDocument to get around needing to write the int() function twice.  Right there we have a problem because we have two classes that call each other.  It just makes the most sense that both of these classes have the same key functions and members in the abstract version, and the rest of the key implementation in the concrete classes.  It makes for a much cleaner implementation at the cost of hereditary chaos.  🙂   Truth be told I find it awesome.

The second major question that I should directly address is, why do I just not save the DashBoardConfigurationDocument bean in the application scope? Basically I am a control freak wanting to do crazy things.  No….  I assure you that I have a reason.  Let’s look at lazy admin A and multi-tasker admin B.  Admin A makes a single change, directly in the appScoped bean before going for coffee, and admin B gets a phone call after doing the same.  Neither are finished with their changes, neither of them had to save the changes explicitly, yet both of the have a change that is already potentially putting the application in an unstable state.  Baaaaaaaddddd  vooodooo….  baaaaaaaadddddd.  For this reason, I also like to separate my editing logic from my global configuration logic. Additionally, I can have XPage UI specific logic in the viewScoped class without feeling guilty about caching stupid spam members in the appScope bean.

I can use this pattern as often as I want, and I can be sure that I do not forget anything.  All of my field names are saved as final strings and I can use them in other sub-classes if I need to.  I can even decide later that I want to override my save function in another bean to get SQL support or whatever. It is just clean, and I like clean.

After taking some time to explain a lot of this to the apprentice, we dove into Expression Language and getting some good old binding done.  It worked like a charm…. almost.

This goes into another crazy use case.  I only wanted one configuration XPage.  I have an enum where I have specific configuration keys saved.  These values are then presented to the user in a combobox where a specific configuration key can be selected, and the document is opened, and the information is displayed.  We did this with Expression Language.  The ComboBox held string values representing each of the values in the enum, and the bean had the necessary getter and setter to take the string and set the enum value for that document.  This setter also triggered a process whereby the configuration was fetched and the values were refreshed.  It was a thing of beauty until we realized that the values were not being refreshed on the XPage although the values in the bean were being refreshed with the contents of the NotesDocument.  It took us two hours to figure this issue out.  The correct getters/setters were being called, the init() function was being called, the correct document was being retrieved, and the values were correct in the bean.  Why were they not refreshed on the XPage?

First off, I thought it was a simple refresh problem.  The errorMessages control displayed no errors, and I thought that it was just a simple case of needing to either perform a full refresh, a refresh directly on the component, or some other crazy thing.  We messed around without any success.  In the end, this was one instance where a simple EL expression just was not enough.  We saw that the EL expression was calling the correct methods;  the onChange partial refresh on the page was working correctly.  My suspicion is that the partial refresh was being performed faster than the methods called by the EL expression.  We took out the EL data binding and instead implemented a SSJS function calling the setter method for the configuration key.  When we did this, everything worked as planned.  We also now have one page that can be used for multiple similar configuration documents that can easily be extended without changes in the design.

Lesson learned:  Java is awesome, EL is cool, but ssjs still has to be used for certain things.

XPage Java dev Tutorial 2 – AdminPanel prt 1


 

General Feels and Gratitude

Hello Everybody!!  Today I am really happy to present my first NotesIn9 video!  It was really great being able to make this sub-series and be a part of the tutorials that really got me going with XPage development.  I want to thank David Leedy for giving me the opportunity as well as thank my employer for allowing me to use what I learn on company time to create content for others.  This is turning out to be a hobby that is really starting to take off.  Again, I have to thank twitter, David Leedy, and Paul Withers for sharing my first blog posts and spreading the word that this blog is out and about.  I would not be so into this right now if it were not for the fresh feeling of being able to do good for others and not just use this website as my own knowledge repository. Thank you to those who read these posts, watch the videos, and of course who retweet, comment and like what I am doing.  All that really makes it easier to stay motivated.


 

Down to Business

What is this mini/sub -series all about?

As you may or may not be aware, everything that we are doing here has a basis in my day to day development.  I started out using java for my xpage dev about a year ago (I had already used java extensively in my daily programming, just not in combination with XPages for which I mostly relied on JavaScript.) To really get started with Java, I decided to make a utility package that included everything that I needed to know to really get going.  I started off with making a sessionHelper.  I did this because I needed to learn how to get a session object.  I did not know anything about the variable resolver at first, and I also had a lot of trouble finding resources to figure it out.  After I managed that, I moved on to one of the most important features of all databases that I develop, the Admin Panel.

The Admin Panel is (currently) a custom control where I store tables that show the variable names and values of all objects located in the applicationScope, the sessionScope, and the viewScope.  I do not waste my time with the requestScope, although I suspect that such a thing would be possible if I found a use case for it.  Additionally to this, I show another HTML table that presents cookie information.  This is a part of another module that I call “the cookie helper”.  Finally, a larger and more complex table presents logging information to the user with a particular role.  The custom control contains functions that erase the variables located in the scoped maps (although I have not made that work without causing errors) add string variables to the scopes (not very helpful, but cool), and delete logging information shown in the debug section of the admin panel.  A great feature of the debug table is that there are six different error levels that all use different row styles.  This allows the trace information to be shown in a more understated way, and fatal errors to pop out like a sore thumb.  Later, I turned this same functionality around to store and show entire session logging documents in a logging database.

Today we are going to look at how to build your own Admin Panel using information built, retained, and retrieved in Java.

How is this sub-series built?

Due to time, I broke this down into three different parts.  Since I am actually coding on camera as much as possible, it is taking a while to get through it.  I am doing this so that you can see where I have difficulty, so that you can hear me commenting as I go, and because my entire concept in this Java Tutorial is to program an application from start to finish together.  When it is just taking too long and when I do not have a lot to say, I do edit out some parts.

Part one (this part) is going to be a basic introduction.  We do, for this one time only, go through a PowerPoint presentation where I go into what we are doing, the three parts, and some basic information about myself.  One thing that changed from the making of the PowerPoint slides to the videos is that the concept had to be changed from asking question before the recording of the next part to all the questions being asked in comments and through email and me answering them in a written format or in a fourth video.  It makes sense for numerous reasons, but we do not need to get into that.

Part two is primarily Java construction.  Although some Java classes were started in the first part, in part two we really get going.  I do not want to spoil it too much here, but we basically finish up all the beans that we are going to need for the custom control.

Part three is being my slight PITA.  I recorded it once already, but am not pleased with how it turned out.  I really need to stop doing my recordings after 1 AM.  I am going to be deleting the changes and redoing the video.  Part three includes registering the beans that we already made with the faces-config.xml file and designing our custom control.  We go into how to add a few custom properties, although we admittedly do not go into every detail that you could possibly go over.  There are numerous NotesIn9s if you need help specifically with Custom Controls.

 Why is this a sub-series?

This is a sub-series because it is a small part in the project that we are building together.  I was trying to find some way to make videos for this page’s main tutorial while also making content for NotesIn9.  I decided that it would be awesome to make some of the content that can really stand alone its own sub series and give them to David Leedy.  For those who follow along here, it should seem pretty seamless.  To those who watch the NotesIn9s, it is my hope that you can watch that content and get what you need without too much knowledge of what we are doing in the rest of the tutorials.  At any rate, I am posting the code on this website.  That being said, without any further ado- I give you the video, and following the video, the code!  Enjoy!


Video

Find the video here!


Code

COMING SOON!

XPage dev Why Java

I recently received a comment that I would like to address:

Hey Greg. I watched the tutorial and followed it quite easily. My biggest block to Java is why should I use it? I am having difficulty understanding what the great benefits are, other than getting skilled in a new field.
I also am perplexed as to why, for example, would I need a session helper when I have session info with my standard XPage. Another thing, having an object of a Domino document and using the object to read and write fields back to my source data seems rather tedious seeing that I can read and write straight to my source data?!?
I’m missing something for sure because everyone seems to be wanting to go the Java route.
If I had 1000 users open an XPage application and view a document then why create 1000 objects, one for each document and session helpers. Does that not take needless memory?

But I am looking forward to the next tutorial!!

There are a few questions in here that I find good to ask and that can cause some debate in the professional world.  I will start off my answering my part to the question why.


Why use Java instead of JavaScript?

Some people will say, “Use Java!  Why?  Because .. white noise………”

I am sure that all of you have heard the “Because it is faster”.  Faster for whom?  The computer?  The difference is negligible.  In a test done by a colleague, iterating over a view with 75,000 documents, touching every one completed only about two seconds faster when done in Java as opposed to JavaScript (14 seconds rather than 16 seconds).  I say, if you are doing something in the real world that needs to touch that many documents and the user is waiting on you, then you are doing something wrong!  I do not care if that is done in JavaScript or Java.  Now, keep in mind that he was doing this for a test and to see if there was a measurable difference.  There was.   We thank him for doing his tests and for sharing the results with us! But to that end, lets make our first point:

Use the correct tool for the correct job!

First off let me say that I find this debate stupid.  It reminds me of the debates that I had to listen my friends engage in regarding their preferences to C# vs Java and Windows vs Linux or Mac.  Every task has its goals, and every goal has its best practice and optimized structure and technology.  If you are working on a project that will be small, used by few, and only requires a few pieces of JavaScript code, than maybe it is best to use pure JavaScript in your implementation.  Likewise, if you are going to be building an application that has code that focuses on customizing the controls, then that too might be best to be done in a pure JavaScript environment.  JavaScript simply is the tool for those jobs.  If you are going to be working on code that needs to be well documented, used by others, maintained by others, or shared between applications or one day may be shared by applications, then Java is the way to go, no doubt about it.  It just is the tool for that job!  Likewise, if you have to build code that handles jobs and working with many documents, then the code does not belong in either of them, but rather run in [scheduled] Agents.*

I know that the separation of Business Logic and UI Logic is another big push the separation of JavaScript and Java.  To that I say bullocks.  Because of the JSF infrastructure and the ability to change existing HTML generation code into actual controls, Java might indeed be the best place to store your code rather than using JavaScript to generate the code for <ComputedField>s set to HTML.  Granted customizing those controls must still be done using JavaScript, but depending on the use case, you might even have some of that logic kept in a Java Object.

Yeah, but, Why Java

Maintainability, Readability, Usability.  There is no better way to put it.

Think back to another Blog Series I did when I discussed the issues I had with debugging JavaScript.  I mentioned that the stupid error

if(apples = oranges) doSomething();

as opposed to:

if(apples == oranges) doSomething();

and that this was an error that should be caught, but never is.  This is my number one reason for pushing for java.  The stupid errors that can cost a great deal of debugging time are caught by the compiler and are easy to fix.  Another case, you have a larger function.  You are dealing with many different kinds of objects, and you accidentally try to set a document object to a documentcollection object.  In JavaScript, you will not discover this error until you run the program and you get an error that states that the desired function is not available, or you end up debugging because it manifested itself in some other way.  In java, a variable declared as a document can only be used for a document.  The error will never happen.  My first reason for pushing for Java in XPages is Usability.  You are not going to have as much code that contain errors and his hidden or uncaught by a compiler.

Another reason why I say usability is improved with java is the tooltips/inteli-sense or whatever else you want to call this feature.  If the javadoc for the API you are using is connected in the build path, you are able to look at detailed information provided by the developers, assuming that they maintained their javadoc well.  This also goes for the javacode that you write.  Of course you have the ability to write and look up comments in SSJS, but they do not show up like they do in Java.  If I am wrong about that please tell me, but I do not believe I am.  In Java, you have the ability to simply use shortcuts to import packages and classes using the inteli-sense, keyboard shortcuts, and see what members and functions are available in those classes whether they are known to designer or not.**  You write a class, that class is known by the inteli-sense. It is that simple.  It may not seem like a lot, but when you can quickly see what code does what, import, and comment all in one place, then why would you not want to?

Readability is also highly improved.  Lets say you have your classes all defined and you are using them.  In JavaScript, you do not have the ability to click on the method being called and have that method opened up in designer, at least not when those functions are defined in two different files.  With Java, this is not an issue at all.  Click on the call, file opens right to the spot in question, time is saved, finish.  The reader is just put into a position where it is efficient to go through your code and figure out what to do.  Another thing: in that same series I linked before, I mentioned that embedded JavaScript function calls are not listed in the outline view.  In Java, this is not the case.  All members, functions and embedded classes and their members are listed in the outline view thus adding to the readability of your code.

Because you are writing your code in a place that makes it readable and because it is in a very usable format that will not require a lot of learning about your code (ie reading your script files to learn how to use it and instead getting the comments via the javadoc and inteli-sense), you are saving time of future programmers who are using your code.  You are also saving yourself time when you need to change something 16 months later and do not remember what you did any longer.  You are also creating something that can be added to later!  Let me explain this.  By saying that, I am not referring to adding some code at some point down the line.  If you want to do that, than JavaScript is completely reasonable.  I am talking about finding out that there are these few classes that you use over and over and over again.  You have to port them into every database you own.  By having these classes in java already, you are in a position where it is not that expensive to port them over into OSGi Libraries that can be put on the server as globally available beans.  If you have them in JavaScript, you also need to translate them.  Furthermore, if you have an HTML generation function that you use for a computed field that displays html code, then it is not so far off to take that code one day and make a real JSF control out of it.  Likewise if you decide to make that code into a control, you can share that control via an OSGi extension.  Overkill?  Nope.  You are simply writing your application in a way that allows for future sustainability.

Why Java?  Not for the negligible speed increase, but for Maintainability, Readability, Usability.


Why a SessionHelper object?

This question refers to a video that I posted as a part of my Java XPage tutorial.  I created a Java object in order to access the session.  First off, I just would like to mention that this is primarily a class to assist other Java classes and not JavaScript.  I most likely would not put this into the faces-config file and register it as a bean, although I have done so in the past.  As the object stands right now, it is merely a list for static methods.  This is a poor candidate for a bean as an instance of the class is not necessary to use its functionality.  Get it when you need it, otherwise leave it.  The only functionality in this particular SessionHelper class that might be helpful down the line in JavaScript is the getODASession().  Otherwise, if I remember correctly, ODA creates a bean called “openSession” that allows you to access the ODA session object simply in JavaScript without any sort of self made utility class.  Even then, you could always access the ODA Factory utility.

There will be some who will say that my approach to getting a session in an external class is a bad idea.  I can only say to that that I have not had any issues getting the session or the database from other helper objects and methods.  An absolute no-go would be getting the objects under the database object.  Your cleanup would either be messy, or give you NotesExceptions depending on how you went about it.


 Java and Object Retention

Beans and the scoped maps are there for a reason.  Lets assume we have an external database that holds language information: labels, keyword lists and so on.  A typical approach from a few applications that I have taken over require that each form retrieve its own values using @DbLookup to fetch them from the source DB.  In XPages, this is dumb.  Likewise, this would be an asinine thing to do for a computed field or hiddenText field, it would be a horrible idea to put it into the viewScope, a dumb thing for sessionScope, and perfect for an applicationScope Bean which maintains a  HashMap of document labels and keywords based on some key and correctly retrieved by reading out the sessionScope bean that contains the preferred language information for the current user as read from his user document.  By putting it into the applicationScope in a bean you are creating a way to cache your expensive values and provide a way to reset those values at will.  Of course you can do similar things with JavaScript, but you cannot put a JavaScript “object” or function into the scoped maps.  We will be getting into such patterns later on in the tutorial.


Conclusion

As Scotty said in Star Trek V, “How many times do I have to tell you, the right tool for the right job!”  There are times when Java is the right tool, there are times when JavaScript is the right tool.  Do not let yourself be put into the position of thinking that only one option is correct, but keep an open mind and remember that future sustainability is directly influenced on your programming technique and tool decision.  Don’t get caught up in the stupid debate of which is better.  Do not listen to the “java is faster” crap.  The real reasons go much deeper than any of that, and if you are having performance problems, then there are much more likely causes than the language choice.

Java is also the only place where you can define classes for beans to be used with scoped maps.  Otherwise, you are just saving raw variables into your map and cluttering it up.  Use Java to correctly store and access your information so that it can be readily accessed and executed.  I am a strong supporter of Java use in XPage applications.  I hope that this article has helped explain why.

 

* This is really a case by case decision and must be decided upon specifically when designing the application.
** By “known to designer”, I am referring to basically hard coding the inteli-sense.  I actually do not know how this works and am merely guessing.  Does not matter though.

A few Years of XPage Developement – Part 6

#IBMChristmasWishlist

My XPage journey being laid out before you, I now come to a point where I want to say what we need in order to be more productive in XPage development.

  1. We need a centralized help application.  Forums and demos are good, but they are not good for everything.  We need a central place to get information on every JavaScript, java, and control function.  We need a listing of all methods opened by each control, and should not have to search long, for example, to figure out how to get the NoteID of a selected document in a view control.
  2. The IDE must be optimized.  This must be done in a few ways.
    1. JavaScript errors must be found at design time.  This is pointing to my if-statement error again.
    2. Nested functions must also be listed in the left side panel of the SSJS editor.
    3. All functions must be listed alphabetically in the left side panel of the SSJS editor
    4. Better and complete java-doc documentation of all JavaScript and java functions which can be seen in the IDE, including better parameter variable names when selecting function.  (control+space).  arg0, arg1, arg2 is just awful.
    5. While binding fields to a control, the field names must be sorted alphabetically and not randomly.  It would be helpful if this alphabetizing was done ignoring caps.
    6. It would be helpful if field names while binding data were also available when using a separate database as a source.
    7. Adding to point f, it would be nice if fields from a selected list of subforms could also be added to the binding options, and that it would work more like type-ahead.
    8. It should be possible to completely ignore the drag and drop and often buggy nature of the Design panel.  I would request that the source editor be optimized for direct xml input.  See visual studio to see what I am talking about.
  3. For those projects where the customer does not want to install the openNTF Domino API, programmers must fall back on the IBM Domino API.  For this, a must have would be a detailed documentation on the use of the API, when to recycle, when not to recycle, and the design patterns that they see as correct and incorrect.  Most designers are left completely in the dark and set up to fail when it comes to recycling and memory management.
  4. Setting properties for multiple controls in background documents was very easy.  Highlight a few controls, add some code, make a few clicks and done.  The designer/programmer never had to search through the property window long before finding what was needed.  The following are things to consider:
    1. Too many clicks are needed to configure controls on the XPage.  There must be a way to optimize this.
    2. Support for changing the values of multiple values at the same time.
  5. Themes.  You can create your own theme in theory, but a lot of stuff is based on dojo and, most of the time, to get what you want you need to get into the innards of Dojo theming.  In the case of OneUI, you have to first understand how it all works.  There should be better theme development support in designer.  This should include altering the Dojo level themes as well as stuff on OneUI.  Optimize this, and you will increase designer productivity by a lot and make point 4b a lot simpler.
  6. With the openNTF Domino API, this point is moot, but get rid of the NotesException.  Completely. It. Must. Go.  I need to handle a SecurityException (User does not have sufficient ACL access) differently than a NullDocumentException (document with given key not found in view) and that differently from IllegalStateException (document was removed or recycled). By having one exception for everything and just changing the “Text” variable (an already bad design), the developer cannot really alter the exception handling without parsing the Text first.
  7. Some issues, whether bugs or not, need to be addressed.  They may not be seen as bugs to the creators, but to the end user who only sees something different than what he sees in the client, it is wrong.
    1. Dynamic categorization in views must also be indented.  Building a view navigator is all well and good, but it is not what he sees in the client, and for many customers that is unsatisfactory.  This is true with the dynamicViewPanel, and the ViewPanel controls, and possibly others that I have not tried.
    2. The Calendar control is a nightmare.  It is complex to set up, buggy in IE, and questionable in other browsers.  It needs a major overhaul and fast.
    3.  I am sure I could think of plenty more if I gave myself the time.

Conclusion

XPages are a good idea in theory.  At its core, it offers a relatively easy way to get a Notes Database open to the web quickly while, under many conditions, not sacrificing usability.  It offers solutions to the current Big Data problem where a single Database cannot contain the data it needs to by offering a behind the scenes and invisible way to access numerous databases simultaneously, whether they are Domino, SQL, or a mixture.  The development of applications is a headache at best, and a catastrophe at worst.  XPages show great potential, but lack what I believe to be real support and backing from IBM.  Instead the XPage development community is left to fend for itself.  If we want XPages to succeed in the current market, than it must be able to compete alongside Microsoft Sharepoint, JSF/JSP applications, and other high end platforms.  This includes programmer efficiency, usability, extensibility, lack-of-buggy-ness, and support.  Call me a masochist, but I want XPages to succeed.

A few Years of XPage Developement – Part 5

Fixing River – opentNTF Domino API

Introducing “openNTF Domino API”

This jewel has been created by a few wonderful developers with tier 5 status, XPage God.  (Yes, I skipped Tier 4 for now).  Developers with XPage God status are those that can create content that no developer can live without.  That content can be pure backend, controls, or both.  Up until now, I have only scratched the surface of what this beast can do.  Its main draw for me was that recycling is a thing of the past.  This was something that should have been done a long time ago by IBM itself.  If we do not have to clean up our objects in Lotusscript, why should we have to do so in Java?  But that is only the first of many improvements.  Iterating through documents in a DocumentCollection has become much simpler.  No longer must you use a while loop that quickly turns into an infinite loop that eventually crashes the HTTP task of the server.  Additionally, error handling is improved.  When an error occurs, the information is saved in files on the server, and if an openLog database is located on the server, the error information is saved there too.  No longer must you play with the NotesException class (a horrible invention to say the least, being the only Exception that I know of where the “getMessage()” method always returns null and that introduced the .Text public property).  The uses for this API are too many to list here.

Since the openNTF Domino API, or ODA as I like to call it, is so important, I have since made it a requirement for hxu version 3.  I have also updated all of my internal databases to incorporate that API.  Once I figured out how to get started with it, using the API is the simplest thing to do.  The methods are almost the same as the IBM Domino API.  The biggest advantages, however, are what to do if you have an issue.  The people who wrote the API are active members of StackOverflow.  If something does not work, you are unsure how to get started, or you get some strange error, write to the original authors via those forums, openNTF, or directly and get help.   Ask for such service from IBM, I dare you.

Future Outlooks

This brings us to the current day and the advent of a Tier 4 XPage developer, “Xpage Sith Lord”.  To Understand Lord status, let’s take a look at the current problem produced by all of my internal databases.  So far, I have about 6 internal active XPage applications.  This is not including development and QA copies.  Each of these applications relies on the hxu jar.  Furthermore, key properties are stored in a file under Resources/Files in the database itself.  Certain controls and libraries are being copied into each database instance.  When hxu changes, all applications implementing hxu must be changed individually.  This is starting to become a bit of a hassle.  Those with Lord Status no longer write stuff for a single database, but rather create OSGI plug-ins that can be easily installed on the server level affecting all applications which have those marked as required.  Controls and libraries need no longer be copied and pasted into new projects, properties can be saved in the xsp.properties file or in the domino notes.ini, and everything can be centrally controlled.  I have a long way to go before I can say that I have reached tier 4, but I can say that I recognize the next steps in my progress as a Notes XPage developer.

A few Years of XPage Developement – Part 4

River – Customer Project 2

River is a workflow application where department heads can create new personnel requests and have these requests approved for human resources to later create positions for them.  It is the starting point to a much larger group of workflow applications.  It is used worldwide and is comprised of multiple databases and replicas on 15 different servers.  The major push for an XPage implementation was that they would like to get rid of the replicas in order to prevent conflicts.  Another major draw is that the notes client would no longer be needed and the future prospects of having a mobile version for the ease of workflow operations.

The implementation of River went very well.  The problems seen during the Solar project were almost completely eliminated.  I very quickly developed a few conventions that aided the development and that I will use from now on.  It was not until half way through that I found my enemy, the NotesException and his buddy, “This Object has already been removed or recycled”.

Let’s get technical for a moment and quickly touch upon a few things that I had no idea of before these lovely issues bared their ugly faces.  Let’s start with a story.  Once upon a time, IBM built Lotus Notes.  They used c++ and built an amazing system to store information in a document and notebook style application and people were (more often than not) happy.  Eventually, IBM decided to give the developers another tool to create agents with and created a Java API.  This API acts as a wrapper for the background C++ objects.  I assume Lotusscript does the same.  The programmer creates a Java object lotus.domino.Database and it points to a c++ object.  One issue however is Garbage Collection.  Using Lotusscript, the developer never has to think about clearing memory.  In java, it is necessary.  For someone who is well versed in java, this is a new concept.  After all, the only time a programmer needs to do that is with NIO and IO objects, (Network) Input Output.  Files, SQL database connections, and streams must always be closed at the end of a function.  When XPages came about many years later, this unintuitive API was still being used.  In my opinion, this was a bad call.   Because XPages is built on Java Server Faces, Java was destined to be a powerful force in XPage development.  Since a lot of developers who had worked with Lotusscript never had any reason to go into Java for most cases, it just became very frustrating.  Many were not just learning a new programming language, but were also trying to get a handle on this terrible API.

If you doubt its horror, take a look at a few use cases.  The user has a document open in an XPage.  This XPage has a function that creates a new document in the background using configured templates located in another database with values located in the current.  This background document is then saved to the user’s mail file and sent to the list of recipients as read from other documents in the same database.  To make matters worse, the XPage application is a dashboard application and all information is saved in other databases.  This is a complicated case, but is not by any stretch of the imagination unusual.  In order to solve this, the programmer makes a managed bean stored in the applicationScope to retrieve the templates from the template database.  It takes certain keywords or variables from the current document, searches for the fitting template, stores the UNID and key in a HashMap, and returns the template.  Everything is recycled and it works well.  The second part of the use case kicks in.  A recipient list is returned in the exact same way as the template.  A search with document keywords is made, the UNID and key are stored in a HashMap for future users, the values are returned, and cleanup is completed.  The source database is also recycled, and right there the problems begin.

The first problem can be solved with a little bit of experience.  In this example we have two different Java Objects: Mailer and RecipientTemplate.  The Mailer has a reference to the source database which is creating the Memo document.  RecipientTemplate also has a reference to the source database where these lists are configured.  The Mailer has created the Memo, retrieved the body template, and, before it is sent, has called the function in another bean to get the recipients.  The next line calls the send method of the memo document object and it fails.  NotesException, object has been removed or recycled.  After banging your head bloody on the desk, screaming so loud that the people in the basement call to ask if you are alright, and a coffee break later, you read this blog entry to figure out what happened.

The session as far as I can tell keeps a mapping of the c++ objects.  This mapping works in a few different ways.  In this example, the first mapping introduces the problem that the RecipientTemplate and the Mailer “Database” objects point to the same single c++ object.  When the RecipentTemplate recycles its Database object, the C++ object is also ripped from the hands of the Mailer object.  But, that explains the database, but we are talking about the memo document being gone.  There is also a mapping of the objects stemming from the higher level objects.  When the database is gone, the documents, and the objects belonging to the documents are also either recycled or are unusable.  I am unsure which.    Knowing this, the programmer can move the call to get the recipient list to the very beginning before the memo document is created.  The call to the function is successful and you go home happy and feeling accomplished, albeit with a headache and blood stains on your shirt.

The next morning you come in, you proudly test your function again.  The mail is sent and you proceed to continue to test your page.  You discover a JavaScript error when reading a value from the NotesXspDocument. Since you wrote “xspDoc.getItemValueString(“fld”)” at least ten thousand times by now, you know that it is written correctly, and you test that the xspDoc is null!  How in blazes did that happen?  Well, you recycled the database in the Mailer function.  Because of this, the current document is gone as well.  This is a case that is not always true either, as was the case with the River project. When the source databases were on a different server from the dashboard, recycling the database object in the Mailer classes did not influence the NotesXspDocument.  This was the case throughout all testing.  As soon as the productive system was created and all databases were hosted by the same server, this issue kicked in.  The first solution?  Delete all calls to recycle the source database because it was just too much trouble.  If there was a memory leak, so be it.

Alright, that is not a good solution, so what should be done to fix the issue without risking a server crash?  This was a question that took many hours of research to answer.  I was once again saying that XPages needed to be buried next to Jelly the hamster.  As was with the Solar project, I was frustrated, angry, and just saw no future for XPage development.  How can a framework exist that just introduces more and more problems?  One issue is solved and another shows up.  That is when I found the next little gem released on openNTF.

A few Years of XPage Developement – Part 3

Recovery

Enter Domino version 9.0.x and goodbye to the “Lotus”

The first thing I heard about the newest version of Domino was that the term “Lotus” was banned.  The second was regarding the debugging functionality.  Since that was my single most influential headache, I decided that I would do a bit of testing and build a Demonstration database for the 2013 Connect Comes 2U hosted by holistic-net in Hannover and SOFTWERK in Fürth.  The demonstration was primarily a padawan level database written by a Jedi Knight level developer.  Key controls were implemented with default values and simple data binding.  Using the Application Layout and the OneUI themes, a front end was created very quickly.  Because limited JavaScript code was used, the debugger was not an issue at that point, though I did make limited use of it.  The most awesome point about this particular demonstration database was that CRUD operations were performed on data stored in a MySQL database.  Even though the database design was relatively simple, the GUI design itself was impressive and took an unbelievable 2 weeks to implement.

With a few words of praise from the boss, the wonder of a new Domino version that showed great potential, and the desire to graduate to “Jedi Master” status, I took on another project and entered Tier 3.   Tier 3 is marked by excessive use of Java Code and a limited use of JavaScript.  The debate between which is best has been touched upon by many a blogger, and so I am not going to get into that at all, but I will say that the IDE, based on eclipse, is very java ready.  Most of the problems I had with JavaScript were no longer an issue when implemented in Java.  The ‘if’ statement issue, explained under Solar, is caught by the java compiler as are many other similar errors.  If you state that a variable is of type “Document” it can never be set accidently to a “DocumentCollection” value and so on.  This is my primary reasoning behind standing behind java implementation of XPage functionality as opposed to JavaScript.

Once it became clear that I would be given a second Customer XPage project, I started to dive into the Java Code and create my own utility classes.  This single preparation project led to another and both of them were used for the customer project.  The first of these has been titled “holisticXutils” or “hxu”.  hxu is primarily a java jar containing Java helper classes.  It provides shortcuts for working with vectors, strings, and Notes Names.  It provides functionality to help with document locking, logging, development, and property files.  It contains custom controls that help with Scrollable Panels, code logging directly visible on the browser to those with the correct ACL role, and a few JavaScript helpers as well.

A few big strides were taken in this Project.  Most notable, all JavaScript is written in an Object Oriented Style.  One big issue that is currently seen while supporting the Solar project is that all JavaScript functions are written in one massive SSJS library file.  All code is written directly after another.  Adding to the fact that these functions are not listed alphabetically, it takes forever to find what you need if you are unsure how the function was named and cannot use the ‘find’ function.  Writing all code in an object oriented style allows for an application that is, simply put, maintainable, even if it does introduce some problems of its own, i.e. nested function calls are not listed on the left side panel.

Version 0 of hxu was primarily testing and getting comfortable with the IBM Domino API.  This is not easily done for someone who only ever coded in LotusScript.  If you type in “View” in the IDE, you get a View object in many different packages.  It took me a while to figure out that the correct one is in the lotus.domino package.  Getting a handle on the session was also not very intuitive, but simple now that I did it almost a thousand times or more.  Version 1 was the first version used in an actual project.  It did not assume that the extension libraries were installed, but it did require OpenLog from Paul Withers.  (Available on openNTF).  Thanks Mr. Withers!  Version 2.x expanded the Logging functionality and required an update to the databases that implemented version 1.x.  It included Logging into my second preparation Project, holistic XLogger, or “Xlogger”, or “hxl”.  We will have to get into version 3 in a moment.

One huge problem that I had was logging.  I successfully used openLog and the tools provided by Mr Withers to track uncaught errors, but this functionality did not go deep enough for me.  I wanted    the ability to document each user’s session.  I wanted to know what browser he used, which version, and a list of all things done so that I can easily diagnose any problem that arose during productive use of an application I create.  Logging things to the Server log is a questionable practice at best, and an absolute no-go at worst.  I had already implemented all of this in my debugging custom control; all I needed was a way to expand this functionality to save information programmatically to a single document created for the user’s session.  With a quick update to hxu, it was possible to create a single logging function that saved data in multiple places at one time.  These places can also be configured using designer in the appropriate .properties file.

After these two preparation projects – hxu and hxl, I felt confident in my ability as a Jedy Master level XPage developer.  I had all the tools I needed to be successful.  Logging, Java, JavaScript, Extension Libraries, I was a dog on the track waiting for the gates to open and a final “do it” from men in charge.  The Customer Project River was given a go and off I went.

A few Years of XPage Developement – Part 2

Solar – Customer Project 1

Customer project 1, nicknamed Solar, was a solid tier 2 project.  We can refer to tier 2 as “Jedi Knight” level.  As a “Jedi Knight”, the programmer will use not only controls, but will also write some JavaScript code.

Solar was a very large project.  It was to serve as a single entry point for many other databases which all used the same design template.  Beware of the false sense of implied simplicity one could extrapolate from the phrase “same design template”.  Due to the configurable nature of the source databases, the individual source databases were as different from each other as the planets in the solar system, hence the project name.  Style sheets and a bit of JavaScript can change the style class name of a control, but this was configurability to an extreme. It included views created by script, language configuration, keyword configuration, configurations for configurations, too many for a Jedi Knight developer.  But let’s dive into the problems, take a look at what tools were available, and the details of the Solar experience.

Solar was built on Domino 8.5.3 and an appropriate version of the Extension Libraries.  The Extension Library was available for a little while, yet still in its infancy.  Many controls either did not work properly, and some did not work at all.  As is seen even today, if you want to build an application using XPages without the Extension Libraries, most of the controls and functionality that a customer and end user take for granted are not available.  Dialogs that are not self-implemented are not available; there are no value picker dialogs, no name pickers, or “advanced” controls.  Every view shown in an XPage basically needs to be constructed in the XPage using the View Panel or DataTable controls, and in many cases, those XPages must be updated if the background view changes.  There is no “one size fits all” approach.  XPage development without the Extension libraries?!?  NO THANK YOU!

Besides the ever present absence of what Classic Notes developers will call “standard controls” in the IBM installation, there were many other hurdles to overcome.  The most hindering of them was lack of developer help material.  Everyone was new to XPages and a good centralized help center did not exist.  The Designer help was no help at all, pun intended, and there was no home forum for designers.  Of course there were websites out there that to this day offer a place for people to ask questions and get help if they need it; however, unlike the current StackOverflow forums, you were lucky if your question ever received an answer and if it did, it almost never helped.  Being the first of the company to take the plunge into XPages, there was no person locally who could take a look at code and tell me what I was doing wrong.  Partner companies rarely had it better.  The developer was on his own in the early day of XPages.  There was no Designer based Help application, no good forums, no fellow developers, no real documentation of the basic IBM libraries or the extension libraries, and very little more than demonstration databases provided by IBM (Team Room template) and openNTF (numerous applications delivered with the ExtLibs).  The problems of figuring out what to use and when to use them were only the tip of the problems back then.

The more interesting problems happened when you did not know what the problem was, only the symptom.  This is a problem that exists to a point even today, but not nearly as bad.  Domino 8.5 did not come with a debugging option for JavaScript.  This was a nightmare and a big reason why I refuse to develop XPage applications for any server below Domino 9.  I will not do it.  (Truth be told, if the boss says do it, you do it….)  Not having a debugger is one issue in and of itself, but it is made much worse with the implementation of the JavaScript coding windows of the IDE.  When you write JavaScript code in designer, it is only slightly better than writing the code in notepad.  Certain errors could and should be found.  For this, let’s look at an example.  If you want to test that apple equals oranges then do something with pears, the code should look like this:

if (apples == oranges){  pears.doSomething();}

but you write:

if(apples = oranges){ pears.doSomething();}

a very strange thing happens.  First off, this error WILL NEVER BE FOUND AUTOMATICALLY. It will not be spit out as an error during run time, or at design time.  It will simply and quietly replace the value of apples with the value of oranges.  This sort of mistake, which can happen unbelievably quickly when trying to type functions in a semi quick way, can cost hours if not days of searching when you do not know that it happened and can just see that the result of the function is wrong, if you notice in the first place.  Without debugging functionality, you can kiss your week goodbye.  The only option you really have is to spit out values to the server console and hope you eventually find it, or have a bunch of interns read your pages and pages of code to see if they find anything.

If that was not enough, when you create a SSJS library, all un-nested functions and variables are written on the left side panel.  This is the same functionality as seen when writing a Lotusscript library and is nothing new.  One huge disadvantage is that these variables are not sorted alphabetically, but rather in the order in which they are in the file.  This is highly annoying.  During development, the developer must write the code in alphabetical order or risk not finding the functions later.

Because of these limitation, the Solar project ended up taking roughly four times longer than planned, and even was in danger of reaching or exceeding a year of development time.  At that point, I did not want to touch another XPage project as long as I lived.  I was frustrated, angry, and dumbfounded.  IBM had sold something as “easy” and “quick” and “the future” and all I wanted to do was bury it outside next to Spot the cat.  We had seen that it was not easy, it was not quick, it was expensive for the customer, and if it was to have any future at all, things would have to change dramatically in the next version of Domino.

A few Years of XPage Developement – Part 1

Introduction

A few weeks ago, I was given the task of writing a blog entry for holistic thinking about my/our XPages experiences over the past few years.  It was thought that this entry would be, perhaps, a page or two in word and that would be it.  Boy was I wrong.  It just so happens that there is just a ton of stuff to write, a lot of ideas to get across, and, for that reason, I am separating this into a few different entries.  In this series we are going to be discussing how my XPage experiences have changed over the course of the past four years and how I expect them to continue to change.  We will be discussing two major customer projects (no worries, it is all anonymized), the XPage learning curve, and the major tiers of XPage developers.  We will look into the problems that we came across, my reactions to them, and how they were solved, or not.

The early years

I had my first taste of XPages in 2011 working as an apprentice at holistic-net.  Enthusiastically, I built my very first web application which was designed to maintain my daily records for the IHK (Industrie- und Handelskammer).  I had already built numerous Classic Notes applications by that point including a Lotus Notes Client version of that same application.  I was pretty satisfied with how quickly I could pick up Lotusscript, the design patterns, and all of the other stuff one needs to be successful with Lotus.  As one old commercial stated, if you can work with spreadsheets, you can build a Lotus Notes application.   This started to be the case with XPages as well.  After all, putting some controls on a page and binding them to a background document or view is not that hard, right?  RIGHT!  Well, almost. Let’s break down XPage development into five different tiers, expanding on a blog entry done by Niklas Heidloff where he described 3 classifications of XPage developers.  A tier 1 developer, affectionately referred to as the “Padawan”, will take a few controls and throw them onto the XPage and call it a day.  My Records Application was little more than this.  It had simple data bindings and was not much work.  It was simply reusing existing controls in their most basic form.  Nothing from it could be reused in other XPages, almost no JavaScript was used if any at al, and absolutely no Java was written either.  The only good thing about this project was that I was able to get accustomed to the standard IBM controls.  I guess every developer is going to have to have such a project where they just get used to standard controls and try to do something with them.  This particular application was used for a grand total of 2 weeks before going back to classic notes and was an absolute failure.  It did however form the foundation for Customer Project 1.