Archive for the ‘opensource’ Category

Building and Maintaining Technical Documentation – Markdown with Gitbook Tooling

Documentation, the word that brings cold sweats to techies far and wide, makes product managers roll their eyes but is the one essential ingredient in aiding adoption and usage of software tools and services.

A key mantra for software development and delivery is “documentation, documentation, documentation” while agile purists will take “Working software over comprehensive documentation” to mean that no documentation.

Obviously in today’s world the expectations for clients, is that software is key and must evolve quickly to meet changing needs measured in weeks, and not months. This fast paced change actually highlights the importance of documentation, but places pressures on it to evolve more rapidly, be easier to use and understand, while maintaining a trail of changes being made within a rapidly changing environment.

Many formats have come and gone over time, plain text,  HTML Help, Windows Compiled HTML Help (CHM), Oracle and Sun Help, Eclipse Help, Flash help not forgetting PDF and MS Word documents for printed manuals. The common practice was to use a single tool to develop the help that compiles into multiple help formats.

Fast forward and the model seems to remain the same, however the challenge being what markup language to use to enable generation. In comes Markdown (https://en.wikipedia.org/wiki/Markdown) which aims for readability (JSON, YAML) so that users do not need to know markup but provides a way of formatting with simple conventions. Interestingly at the time of writing this even Whatsapp one of the most popular chat clients uses markdown like formatting.

The formatting challenge has been solved but now how to build the content, version control it to keep track of updates, generate the content and share it with the world. The most common tools are:

  1. GitHub pages (https://pages.github.com/) – using a special branch within a GitHub based repository to build online based documentation
  2. GitBook (https://www.gitbook.com/) which provides an excellent editor, hosting, build for generating PDF, online documentation, and mobile format (.epub and .mobi). Just open an account, fire up the editor and you are good to go

However if you are using private repositories and need to keep content internal facing, then you need to pay quite a bit for GitBook or jump through multiple hooks. However GitBook fortunately provides a command line client which can be used in this case to build the documentation which is then distributed using internal channels.

The steps to setup a local Gitbook environment are:

  1. Install npm
  2. Install Gitbook cli by typing the command below
    $ npm install gitbook-cli -g
  3. Setup a git repository for the project and add the following files:
    • .gitignore – include the _book directory in which the book will be generated
    • book.json sample below
      {
        "title": "My Book",
        "description": "Description",
        "author": "Author Name",
        "gitbook": ">= 3.0.0",
        "structure": {
          "summary": "SUMMARY.md",
          "readme":"README.md"
        }
      }
    • README.md the default information on the book
    • SUMMARY.md the Table of contents for the book, which may be in different directories recommended is a docs directory
      # Summary
      
      * [Introduction](README.md)
      * [Chapter 1](docs/chapter1.md)
      * [Chapter 2](docs/chapter2.md)
    • package.json – contains build commands for the book
      {
        "scripts": {
          "docs:prepare": "gitbook install",
          "docs:watch": "npm run docs:prepare && gitbook serve"
        }
      }
    • The final project structure looks like

      Screenshot 2017-03-27 11.13.13

      Example gitbook project structure

  4. You can view the book locally by running the command below which starts up a server running on port 4000

    $ npm run docs:watch

  5. Commit the contents to your git repo

I have to say that I love the GitBook Editor which works way better than my IDE, so after commiting the intial files, I fire it up and open the directory containing my project so that I can edit the files from there. Obviously I lose the ability to put high quality comments on what has changed in the files without jumping back into my IDE or git command line, but the sacrifice is currently worth it.

Additional Steps to Generate ePub and PDF

  1. Install Calibre (https://calibre-ebook.com/download) which provides the ebook-convert utility
  2. Add tasks to the package.json as below:
    {
      "scripts": {
        "docs:prepare": "gitbook install",
        "docs:watch": "npm run docs:prepare && gitbook serve",
        "docs:generate-epub" : "gitbook epub ./ ./_book/mybook.epub",
        "docs:generate-pdf" : "gitbook pdf ./ ./_book/mybook.pdf",
        "docs:generate" : "gitbook build && npm run docs:generate-epub && npm run docs:generate-pdf"
      }
    }
  3. Generate epub by running the command

    npm run docs:generate-epub

  4. Generate pdf by running the command

    npm run docs:generate-pdf

  5. Generate both epub and pdf by running

    npm run docs:generate

UPDATE: More information on the GitBook Toolchain  can be found at https://www.gitbook.com/book/gitbookio/docs-toolchain

UPDATE2: Added steps to generate epub and PDF documents

UPDATE3: Discovered that the process has a name – Documentation Driven Development, which is pretty interesting concept … https://twitter.com/brnnbrn/status/847197686042312704/photo/1

c8hzw6qu0aayg2r

 

Advertisements

Alternate Approach to Legal Independent Election Tallying

The Uganda elections are more or less over with less than 6 hours for the Uganda Electoral Commission (EC) to announce the results for the presidential elections.

Given all the time on our hands, with no social media, the team at Styx Technology Group designed the following alternative approach to independent electoral vote tallying for future elections that provides inbuilt mechanisms for audit and verification of results.

The primary data sources for the process are:

  1. Official EC list of polling stations and voters per polling station
  2. Photos of the signed election tally sheets from each polling station. To ensure that the photos are not tampered with and provide an audit trail:
    • Each photograph has to be taken with information on the camera, the GPS coordinates of where the photo was taken, date and time when the photo was taken which is available in many cameras that share it using the Exchangeable Image File Format (EXIF)
    • Two separate photos of the tally sheets have to be taken by different cameras
    • The cameras taking equipment may be registered beforehand to provide validation of the source of the information
    • The signatures of the returning officers and stamp must be clear and visible in the photo

The architecture for the technology solution is as follows:

  1. Web based solution accessible via any browser. Due to poor Internet connectivity in many areas of the country, an Android app would be provided to assist in data collection, then data sent once the user gets into an area with Internet.
  2. The field officers who capture the photos would also be provided with an option of entering the candidate vote tallies.
  3. In the tallying center, candidate vote tallies are entered from the photos received and vote tallies entered by data clerks. In order to reduce errors the following approach would be used:
    • The clerks are randomly assigned photos as they come in
    • The tally for a station must be entered correctly by two separate data entry clerks, then approved by a supervisor. This process is formally called the two-pass verification method or double data entry.
  4. All correctly entered data is shared with the rest of the world for download and analysis.

This system is mission-critical having to be available for the entire vote counting period of 48 hours,  so the architecture includes the following paths for data collection:

  1.  Multiple access IP addresses and domains for the website in case some are blocked off
  2. Any data collected via the Android app can be sent via email to a dedicated tallying center address. To ensure that only data from the app is received and not changed in transit, encryption is used.

The inspiration came from a quote by Ghandi “Be the change you wish to see in the world”, disproving the myth that there is no local capability to design and implement such solutions and most of all that such solutions have to be complex.

Looking forward to hearing your thoughts and suggestions…

Tech Tip: Websites on Github with Jekyll

We have been facing a challenge on how to manage the Styx Technology Group corporate website without having dedicated resources to host, deal with security, updates and maintenance. Having been in the website business for over a decade at the turn of Y2K,  having an easily maintainable solution was key for the long haul.

Enter GitHub pages (https://help.github.com/categories/github-pages-basics/), of course any decent software developer and team are using some form of version control, but the beauty is that it allows projects, users and organisations to have self hosted sites that are managed through a repository.  That in combination with Jekyll (http://jekyllrb.com/) supports content management without a database.

To further speed up our development process, we leveraged existing themes for the website look and feel that were close to our need, while keeping the site design very simple so that we focused on solving the problem at hand.

It was suddenly music to our ears as the team did not have to leave their IDEs (random plug for Jetbrains IDE tool suite that we have standardized upon)  in order to make updates to the website. Just create a new markdown document, verify and push … And bam!! the updates are done, version controlled and all, no more hassles with database configurations …

Need help getting your website up and running in such a fashion do shoot us an email at consulting at styxtechgroup dot com and we shall be happy to help and engage…. Have a great weekend

OpenMRS Implementor Series 101 – Building a distribution package

I am not sure where this will end, however it is my way of giving back to the OpenMRS community that has provided an excellent learning platform and great support to me over the last 6 months, I have been using the software tools.

My intention is to share what I know, as a way of organising my thoughts and sharing ideas as I figure out where to add this to the OpenMRS community documentation, where it surely belongs. The flow of each of these posts will usually be to answer a very specific question that I have asked in the past, and where I think I have figured out an approach that seems to work.

UPDATE December 31, 2015: The new home for this blog post is  on the OpenMRS Wiki at https://wiki.openmrs.org/pages/viewpage.action?pageId=93358939 all further updates will be placed there

Problem Statement: How can I get an OpenMRS distribution that has all the modules that I need?

New to OpenMRS – What is Possible?

If you are new to OpenMRS and would like to see what is possible, then download the OpenMRS X.X Standalone Edition from http://openmrs.org/download/ which contains a pre-configured reference application with sample data that is ready to run in a single command.  This is note meant for production use as the embedded components are not robust enough for the growth in data collected.

Enterprise Installation like Reference Application

If the reference application provides all the modules that you need, the next step is to set-up n enterprise ready installation using a dedicated application server & database in which the Platform x.xx WAR (for Enterprise) can be installed. The modules for the reference application above can be downloaded as a zip file from the standalone edition above.

Full Control of Enterprise Installation

The last option is the most complex, and is for those implementors who want to install the enterprise edition but have control over what modules are installed or even what versions. In this case the OpenMRS distro platform is your best choice (https://github.com/openmrs/openmrs-distro-platform) which allows the building of a custom WAR file that includes the modules that you need.

The modules are defined in the pom.xml file and are accessed from the OpenMRS Maven Nexus repository (http://mavenrepo.openmrs.org/nexus/index.html#view-repositories;modules~browsestorage).

An addition to this project is for modules (pending review) but currently available at https://github.com/ssmusoke/openmrs-distro-platform is the ability to include custom modules (*.omod files) by dropping them in a folder called modules.

Please do share your comments on this style and let me know where it would fit best in the OpenMRS community documentation

 

Alternate Approach to Voice Price Wars in Telecom Industry in Uganda

My colleagues at Styx Technology Group are looking at alternate approaches to telecos in Uganda to increase their ARPU (Average Revenue Per Unit) a metric for revenue from each customer, instead of the current price war tagged to 3/= per second (US$ 0.1 per minute).

  1. Accept that voice is now commodity, being pushed further out by VOIP for both regular users and business, due to the improving Internet connectivity both via fixed and mobile connections. There is no longer a competitive edge to having cheaper voice, the revenues are fixed and can only go lower
  2. Bundled services: Currently there are separate plans for voice, SMS and data, which have to be purchased daily or when needed. The monthly plans have a premium attached, so without looking at the numbers I suspect that a majority of the regular users purchase daily plans as and when is needed. The telecoms can create bundled plans (already existing for voice) to include SMS and data without the hefty premium. Additional incentives can be provided for further discounts when a user pays consistently for a plan for 6 months, without any breaks.
  3. Smartphone Device and Service Contracts: While these are being gotten rid of in the US and Europe, the market in Uganda is ripe for disruption, where smartphones are paid over 12 to 24 months, with bundled services. Obviously the argument here is the risk associated with lending in Uganda, but options include partnering with financial institutions can help reduce the risk profile, work through employers to deduct the costs of the contract directly at source.
  4. Multiple Smartphone Data Plans: This is similar to the device plans above, however this allows the owner of the plan to register additional devices for monthly fee to share the data. This has been common with unlimited plans, and would provide a new revenue stream.
  5. Extending Mobile Money Services: The best service to copy is PayWay with a wide range of devices, and platforms on which to use the service based on what infrastructure the agent has. I would like to be able to swipe my VISA card and transfer money to my account without having to go through the bank interface which tends to be down more often than not.
  6. Bulk Sales of Devices to Schools: The new underlapped customer base, sell more devices to schools get parents to pay part of the costs to push e-education services, why do kids still have to fill Advanced Level and University Level choices on paper forms that can be lost? With powerful tablets in the $50 to $100 range only the telecoms have the clout, network and drive to push this through.
  7. Custom Devices and Services: These are for data collection needs, surveys etc, which can be accessed through third parties but pushing the envelope on what is possible. The key here is flexibility of service, enabling channel partners build and innovate by creating custom services and plans to meet their specific needs.
The telecoms need to think of blue ocean strategies to create new markets, provide ability for others to leverage their platform investments for new revenue channels, leveraging the example of Amazon that has created a multi-billion dollar technology infrastructure business based on solving internal problems.
What do you think?

UPDATE: This blog post follows the same thinking as The Telecom Wars in Uganda – Round 5 – 2015 and Beyond on this blog too

OpenMRS Module Development Learnings – 102

My team mates and I are completing a number of OpenMRS workshops focused on improving our module development capability. This blog post captures the unwritten lessons that we have picked up as a way of giving back our lessons to help others in the community (and will be moved to the OpenMRS Wiki once the lessons have been internalized and consolidated, as that is its natural home).
The approach is based on the following OpenMRS community articles:
  1. Creating Modules – https://wiki.openmrs.org/display/docs/Creating+Modules
  2. Creating your First Module – https://wiki.openmrs.org/display/docs/Creating+Your+First+Module
As you develop the module it is important to keep testing and verifying whatever you do especially when the module involves user interface components, extending the OpenMRS UI. The process we ended up using was as follows:
  1. Setup a module using the module creation command line utility
  2. Downloaded Spring Loaded from its maven repository http://mvnrepository.com/artifact/org.springframework/springloaded
  3. Downloaded and setup OpenMRS standalone from http://openmrs.org/download/ picking a version greater than 1.11.x which supports debugging options
  4. Once the standalone version is installed, open the openmrs-standalone-runtime.properties and add the following variables to the vm_arguments: (should all be one line – separated here for clarity)
    • -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000
    • -Dmodule.development.directory={absolute path to root folder of the module}
    • -javaagent:{absolute path to spring loaded jar}
    • -noverify -Dspringloaded=inclusions=org.openmrs.module.modulepackage..* (note the two trailing dots before the *)
  5. Setup newly the module into an IDE (tested with IntelliJ & Eclipse)
  6. Installed a FileSync utility, RemoteSynchronizer for IntelliJ & FileSync (http://andrei.gmxhome.de/filesync/) for Eclipse
  7. Setup sync for the omod webapp folder into the standalone webapp folder located at WEB-INF/view/module/{modulename}
Our development workflow is smoothened out since any changes to Java classes & JSPs are automatically applied & reflected in the standalone app.
Additional Tips and Tricks
  1. Tomcat – Change the following init params in web.xml:
    • modificationTestInterval from 4 to 0, so that JSPs are automatically recompiled
    • development is true (this is the default)
  2. Intellij – the Java classes are not automatically recompiled on saving changes so there is need to run mvn package -DskipTests
  3. Follow the OpenMRS conventions as much as possible by cloning openmrs-core to see how the design of the interfaces, service layers is done. This will help get access to lots of the magic that happens behind the scenes.
  4. Java 8 will give you trouble, stick to 1. 7 as we did not try 1.6 anywhere.

Mapping Adventures Day 1 – Introduction and Open Street Map

Mapping has grown by leaps and bounds, from the introduction of Google Maps, what was once the ode of cartographers and GIS experts is now available to the common folk like me 🙂

So wanting to learn how to map is one thing, getting the chance to do it is another, until well Fruits of Thought (http://www.fruitsofthought.org/) organized an exercise to update the information in Kabalagala, a local suburb of Kampala the capital city of Uganda.

The agenda was really simple, an introduction to mapping (and what we were going to do), the we were going to go out and collect the data, return for lunch after which we would upload the data we have collected to Open Street Map (OSM – http://www.openstreetmap.org).

The introduction to mapping was a simple affair, the concepts introduced where:

  •  Trace – GPS coordinates for a place, when entering these into OSM the type of feature found would also need to be described
  •  Track – the route taken to a point. The value of this was adding information on tracks, and side roads.

The GPS data collection devices were eTrex Venture HC Garminand GPS Receivers (https://buy.garmin.com/shop/shop.do?pID=8707&ra=true) and android smart phones with Open Streetmap Tracker  a simplified app that captures the GPS coordinates of a single location at a time.

Well then off we were down to the dirty collecting trace points and grabbing data for a few routes. The team I was in was on a mission and within 90 minutes we had a 4km route and 40 points of data. That was the easy part; next step was lunch then uploading to OSM.

First we had to create accounts which was pretty straight forward since it also supports OpenID so I used my Google Account, yes I am a fanboy. OSM requires a GPS Exchange format (gpx) file which was easily downloaded from the GPS receiver unit we were using.

An initial challenge we had was with Internet connectivity as for some reason it was very slow that evening so the upload of a 600K file took forever and failed later, but finally we got it in. Once the file was uploaded we could access the traces at http://bit.ly/KK15ql to start adding more information. This turned out as easy as drag a building and facility type and place it over the trace point, give it a name and details … A baby could do it in their sleep, isn’t that what we all say when we learn something?

Well after all is said and done we need to praise Google Maps for leading the charge, and so did this blogger “In Praise of Google Maps” (http://oleb.net/blog/2012/06/in-praise-of-google-maps/)

%d bloggers like this: