Archive for the ‘web development’ Category

Alternate Approach to Legal Independent Election Tallying

The Uganda elections are more or less over with less than 6 hours for the Uganda Electoral Commission (EC) to announce the results for the presidential elections.

Given all the time on our hands, with no social media, the team at Styx Technology Group designed the following alternative approach to independent electoral vote tallying for future elections that provides inbuilt mechanisms for audit and verification of results.

The primary data sources for the process are:

  1. Official EC list of polling stations and voters per polling station
  2. Photos of the signed election tally sheets from each polling station. To ensure that the photos are not tampered with and provide an audit trail:
    • Each photograph has to be taken with information on the camera, the GPS coordinates of where the photo was taken, date and time when the photo was taken which is available in many cameras that share it using the Exchangeable Image File Format (EXIF)
    • Two separate photos of the tally sheets have to be taken by different cameras
    • The cameras taking equipment may be registered beforehand to provide validation of the source of the information
    • The signatures of the returning officers and stamp must be clear and visible in the photo

The architecture for the technology solution is as follows:

  1. Web based solution accessible via any browser. Due to poor Internet connectivity in many areas of the country, an Android app would be provided to assist in data collection, then data sent once the user gets into an area with Internet.
  2. The field officers who capture the photos would also be provided with an option of entering the candidate vote tallies.
  3. In the tallying center, candidate vote tallies are entered from the photos received and vote tallies entered by data clerks. In order to reduce errors the following approach would be used:
    • The clerks are randomly assigned photos as they come in
    • The tally for a station must be entered correctly by two separate data entry clerks, then approved by a supervisor. This process is formally called the two-pass verification method or double data entry.
  4. All correctly entered data is shared with the rest of the world for download and analysis.

This system is mission-critical having to be available for the entire vote counting period of 48 hours,  so the architecture includes the following paths for data collection:

  1.  Multiple access IP addresses and domains for the website in case some are blocked off
  2. Any data collected via the Android app can be sent via email to a dedicated tallying center address. To ensure that only data from the app is received and not changed in transit, encryption is used.

The inspiration came from a quote by Ghandi “Be the change you wish to see in the world”, disproving the myth that there is no local capability to design and implement such solutions and most of all that such solutions have to be complex.

Looking forward to hearing your thoughts and suggestions…

Tech Tip: Websites on Github with Jekyll

We have been facing a challenge on how to manage the Styx Technology Group corporate website without having dedicated resources to host, deal with security, updates and maintenance. Having been in the website business for over a decade at the turn of Y2K,  having an easily maintainable solution was key for the long haul.

Enter GitHub pages (https://help.github.com/categories/github-pages-basics/), of course any decent software developer and team are using some form of version control, but the beauty is that it allows projects, users and organisations to have self hosted sites that are managed through a repository.  That in combination with Jekyll (http://jekyllrb.com/) supports content management without a database.

To further speed up our development process, we leveraged existing themes for the website look and feel that were close to our need, while keeping the site design very simple so that we focused on solving the problem at hand.

It was suddenly music to our ears as the team did not have to leave their IDEs (random plug for Jetbrains IDE tool suite that we have standardized upon)  in order to make updates to the website. Just create a new markdown document, verify and push … And bam!! the updates are done, version controlled and all, no more hassles with database configurations …

Need help getting your website up and running in such a fashion do shoot us an email at consulting at styxtechgroup dot com and we shall be happy to help and engage…. Have a great weekend

Ruby on Rails for the PHP Web Developer – The Journey

For my frequent readers, you may assume that I have run crazy … now Ruby on Rails with all the PHP experience that you have got! Well I am shocked as well, but its a necessary step in my growth as all of us have to move out of our comfort zones at some time. Well I am going to be moving to an organization that uses Ruby on Rails as their language tool of choice as a project manager. So yes I have to learn a new tool, leveraging my experience to manage agile projects, so the geek in me has to understand the tool chain to enable me have better dialogue with the business analysts and developers.

So I am starting out on a new path, but as I was taught (brainwashed actually) the best way to learn a new tool is to use it. So I am trying to build a not so super secret software solution to solve a business problem I keep running into as a way of getting into the guts of this tool.

Ruby on Rails (ROR) is tough to get running on Windows as it was more a *nix development tool chain but I am taking up the challenge to get it to work with lots of Google searches and the ever present StackOverflow (now its my chance to ask questions). The environment is as follows:

  1. IDE – started with RubyMine for Jetbrains (http://www.jetbrains.com/ruby/) with a 30 day evaluation, then I will see what happens next
  2. Rails Installer by Engine Yard (http://railsinstaller.org/) – the smoothest way to get Rails up and running. The strange thing is that I am using Ruby 1.8.7 as that’s the version being used by an opensource project I would like to contribute to … so well why not stay behind and do some good while I am at it
  3. Ruby on Rails Tutorial (http://ruby.railstutorial.org) by Micheal Hartl free online resouce
  4. Stack Overflow (http://stackoverflow.com) – Q&A site most of the issues that I face have already been solved
  5. Git – needed a private repo so ended up at BitBucket (http://bitbucket.org) by Altassian. Having little knowledge of Git too, and having a lot of problems setting it up I downloaded SourceTree (http://www.sourcetreeapp.com)
  6. Ruby Gems – pre-built functionality

Now in the process of moving the dev environment to a Mac … However after trying to install the different pieces alone, I was advised to use Home Brew (http://mxcl.github.io/homebrew/) following the instructions at http://bit.ly/114X2x1

Part II coming up soon ….

My first Symfony 2 Web Application – The Journey – Installation

Happy New Year, 2013, its finally here … Well in 2012, I should say I had Symfony 2 envy, almost akin to a high school crush, so after suffering in silence for so long I have finally decided to jump in with both feet.

I am developing a simple website which is esentially a set of utilities for my home country Uganda, and the idea is that I can learn a bit more advanced PHP but also opensource it later to use as a teaching tool for other local developers (oh yes, I want to share my knowledge too). This will also give me the chance to see if I still got it in me because I do not touch code in my day job, akin to Neo in the matrix hacker by night.

So what I have I used, well an evaluation copy of Zend Studio 9 (used to use an older version at my work place), Symfony 2 (latest distribution), Twitter Bootstrap, JQuery, Git (I am using a private Bitbucket repo) and well an open Google tab. My first words are wow!! Alot has changed in the world of PHP since I last used it, Composer, Less, Sass which requires Ruby … But hey here goes

BTW I am running all this through a WAMP (http://www.wampserver.com/en/) install again the fastest I could get things to work, you could use XAMPP too but I have never done so. I am using PHP 5.3.13 with MySQL 5.5 on Windows 7 64bit. One question I may receive: As a newbie why are you going for the whole enchilada (Symfony2, Composer, Bootstrap Integration etc) – since this is a clean break I am trying to also get an understanding of the different tools that have been developed, how to use them and what they mean for the “average” web developer to answer the question – are they worth it?

  1. So first step was installing composer, which I did following the instructions here http://getcomposer.org/doc/01-basic-usage.md after which I installed it globally so that I could call it anywhere following the instructions in the answer http://stackoverflow.com/questions/12059397/is-there-any-way-to-install-composer-globally-on-windows  
  2. Next was to install Git using Git for Windows (http://msysgit.github.com/) needed even for local staging
  3. Being a git newbie, I could not get things to work properly to create the project I followed the workflow below: (I stand to be educated on a better workflow)
    • Created a new Git project on Bitbucket
    • Created a new PHP project from Git within Zend Studio – which gets everything configured well
    • Downloaded the Symfony 2 standard edition and copied the files into the new project
    • Followed the instructions here to commit it …
  4. Integrated Twitter Bootstrap using the Mopa Bootrap Bundle ( https://github.com/phiamo/MopaBootstrapBundle) however this required me to install Ruby for the LESS integration using Ruby gems while I initially run into a couple of problems with the installation of mopa/composer-bridge it was probably a connection issue. Also note the following:
    • Running the composer update command requires a console with administrator privileges to be able to create the symbolic links
    • The lessc.bat file in the Ruby install folder needs to be updated to use absolute paths to ruby.exe other than the relative paths in the file
    • Well after a lot of troubleshooting I found out that the ruby gem for Less was depreacted so I ended up using LessPHP (PHP compiler for less in pure PHP) following the instructions at http://isometriks.com/using-less-with-symfony2

Now the initial installation is done, I have my Symfony2 project running with Twitter Bootstrap integrated, it has been about 10 hours of work, but totally worth it as a learning experience

Uganda Web Developer Workshops: Rotary Style – Idea looking for Partners and Direction

This is an idea that has been growing in my mind, and I seem to finally have a handle on it. I have been a professional PHP web developer for the last 12 years, and have gone through the learning and transition cycles from learning a new language from Visual Basic 6 and MS Access to Java/JSP/Servlets to PHP, and developing in the language from direct database access using mysql_query through a custom developed database class into Zend Framework for 2 years with Doctrine 1 and 2 ORMs.

My biggest challenge has to do with the fact that there are no places to go and talk code, PHP/Javascript/Database in Uganda, along with the experiences and challenges facing web development with “need-to-have” practices like:

  1. Refactoring
  2. MVC design for web applications
  3. Version control – branching, merging, version tagging and management
  4. Testing – unit and functional testing, load testing
  5. API development (okay this is pushing the enveloper)
  6. Continuous integration, code quality metrics – complexity, modular development, cyclomatic complexity
  7. Team Style development – PSR 0 and PSR 1 compatibility
  8. Frameworks – Symfony, Zend Framework, Kohana, JQuery, Twitter Bootstrap
  9. Advanced CSS and HTML 5 – style guides, browser targetting, mobile development

I am also looking at doing this Rotary style, 1 hour developer meetings once in 2 weeks, then later once a week, same night, same location.

The bottom line is that we develop the quality of the available pool of web developers by growing a community, having role models, to also put a brighter face on the industry, improve perceptions and make it clear that this is an area that has professional practices. It is a win-win for all involved

Any ideas who has done this before, what were your challenges and trials, who would like to partner on this?

Databud – Startup Weekend Kampala – April 27 to 29

I will be attending my first startup weekend in Kampala, on April 27, 2012 to April 29, 2012 and well I thought that why not share my pitch and get advice on how to refine it. No idea is great unless shared right?

In the absence of #opendata in Uganda, there is a whole lot of data locked up within individual government systems, documents, in non standard formats which needs to be unlocked, the data set free so that it can grow (Data Bud) – the data buds and grow

A picture is worth a thousand words right – below is the whole concept

Data Bud Concept

Data Bud Concept

Comments, additions, advice? Looking forward to seeing ya this weekend

The Poor Man’s Job Queue

Not all software development projects are treated the same, some have access to modern tech Virtual Private Servers (VPS), Zend Server (http://www.zend.com/products/server/), Memcached, Gearman and all the other goodies I can only dream of. You have a box with LAMP, and you cannot install anything else.

This is an example of how we got around a limitation, using available tools. Problem: I have a list of tasks to execute within my application, however I need to ensure that the tasks are executed and completed, but some are more important than others, and the execution may slow down the performance of the box we are running on. Well in this case we were loading 6 different types of XML files which were FTPed into a location on the box regularly, every 35 minutes and had to be loaded in a specific order. This was further complicated by the fact that we had to reload historical data in case of issues (1 weeks worth of uploads ~ 2100 files) without interrupting the current loading processes.

The approach used the following components:

a) Job Queue – based on the Zend Server Job Queue but simplified for our needs (see data model of tables below)

Job Queue Data Model

Job Queue Data Model

b) Queue Loader Script – loads the jobs into the job queue by scanning the location containing the files to be loaded and adds the files to the queue (since the queue is a database table, duplicates are discarded without errors) This keeps this file simple and honest

c) Job Executor Script – reads a message from a queue, reads the message body which contains the file name to be processed, could be made more complex

d) Queue Loader Cron Job – calls the Queue Loader Script to add new files to the queue

e) Job Executor Cron Job – calls the job executor script. This job has no effect if a lock file exists, and is not expired which means the script is valid and running. However if the lock file is expired, it means that the process crashed, so the lock file is deleted, a new process is started with a lock file. Basically this keeps the job executor script running indefinitely as long as there are messages to process. 

Please feel free to leave a comment on what your experiences are with similar problems. 

%d bloggers like this: