Archive for the ‘database’ Category

2 Jun

Making MySQL 5.7 fully compatible with 5.6

Posted by ssmusoke in database, mysql, tips and tricks.

As a developer there are times you may need to use MySQL 5.7 but have it backwards compatible with MySQL 5.6, so that existing applications work as before. This is a collection from multiple sources, that seems to get everything to work fine

The settings below when added to my.ini or my.cnf have this effect

[mysqld]
# Only allow connections from localhost
# bind-address = 127.0.0.1
innodb_buffer_pool_size=6G
max_allowed_packet=128M
innodb_file_per_table = OFF
table_definition_cache = 400
table_open_cache = 400
performance_schema=ON
show_compatibility_56=1
sql_mode = IGNORE_SPACE,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
ssl=0

# general log
general-log=1
log-raw=1
general-log-file=/var/log/mysql/general.log

26 Feb

Crowdsourcing Validation Rules for Uganda National ID

Posted by ssmusoke in database, enterprenur, mobile computing, opensource, software delivery, software development lifecycle, styx technology group, tips and tricks. Tagged: .

I am curious about the ability to validate that the Uganda National Identification Number (NIN) is well formed. However this does not validate that the NIN actually belongs to the person presenting it or that it is correct.

The rules that I have been able to gleam are:

  1. Must be 14 characters long
  2. First character is a letter of the alphabet. C seems to be a common letter – does it stand for citizen?
  3. Second letter is either M or F – male or female
  4. Characters 3 and 4 are numbers, which are the year of birth. Cannot be after 00 since that would make a person below 18
  5. Characters 5, 6, 7 are numbers

How can you help? Which of these rules do not match your NIN? Share any additional patters to build a repository of rules that can later be mapped to programming language validations – Regular expressions and validation frameworks

17 Apr

TechTip: Dbunit Export from Jetbrains DataGrip

Posted by ssmusoke in database, software development lifecycle, styx technology group, test driven development (TDD), tips and tricks. Tagged: , , .

I am an avid test driven development (TDD) advocate nowadays, with a pragmatic slant of course, looking to bullet proof the features that I deliver to ensure that they do what is expected, and work out edge cases.

A big challenge to testing is generating of test data, which is needed to setup some integration test work flows. I have been using Jailer (http://jailer.sourceforge.net/) to generate data from existing tables in a Dbunit format which can then embed in my test dataset xml files.

This is a challenge due to the mapping of relationships by Jailer (a neat feature by the way). So while working Datagrip, the database IDE of choice, we were struck by how to export different formats when looking at a table. This solution would allow us to leverage available filtering and searching features, to nail down the datasets that needs to be exported.

On contacting the support team through Twitter (https://twitter.com/0xdbe/status/853900122828222465/photo/1), the recommendation was to modify the existing XML groovy script to generate DBunit XML, following the steps at https://www.jetbrains.com/help/datagrip/2017.1/extending-the-datagrip-functionality.html

And well an hour later below is a groovy script to do just that can be found at https://gist.github.com/ssmusoke/ca4c55b4e52de97acb99a590644a677f which can be used by creating a new script DbUnit-Groovy.xml.groovy 

The code was not being well rendered hence the move to a Gist

20 Feb

Alternate Approach to Legal Independent Election Tallying

Posted by ssmusoke in alternative approach, database, devops, innovation, internet, mobile computing, mobile technology usage, opensource, responsive design, software development lifecycle, styx technology group, technology, tips and tricks, web development.

The Uganda elections are more or less over with less than 6 hours for the Uganda Electoral Commission (EC) to announce the results for the presidential elections.

Given all the time on our hands, with no social media, the team at Styx Technology Group designed the following alternative approach to independent electoral vote tallying for future elections that provides inbuilt mechanisms for audit and verification of results.

The primary data sources for the process are:

  1. Official EC list of polling stations and voters per polling station
  2. Photos of the signed election tally sheets from each polling station. To ensure that the photos are not tampered with and provide an audit trail:
    • Each photograph has to be taken with information on the camera, the GPS coordinates of where the photo was taken, date and time when the photo was taken which is available in many cameras that share it using the Exchangeable Image File Format (EXIF)
    • Two separate photos of the tally sheets have to be taken by different cameras
    • The cameras taking equipment may be registered beforehand to provide validation of the source of the information
    • The signatures of the returning officers and stamp must be clear and visible in the photo

The architecture for the technology solution is as follows:

  1. Web based solution accessible via any browser. Due to poor Internet connectivity in many areas of the country, an Android app would be provided to assist in data collection, then data sent once the user gets into an area with Internet.
  2. The field officers who capture the photos would also be provided with an option of entering the candidate vote tallies.
  3. In the tallying center, candidate vote tallies are entered from the photos received and vote tallies entered by data clerks. In order to reduce errors the following approach would be used:
    • The clerks are randomly assigned photos as they come in
    • The tally for a station must be entered correctly by two separate data entry clerks, then approved by a supervisor. This process is formally called the two-pass verification method or double data entry.
  4. All correctly entered data is shared with the rest of the world for download and analysis.

This system is mission-critical having to be available for the entire vote counting period of 48 hours,  so the architecture includes the following paths for data collection:

  1.  Multiple access IP addresses and domains for the website in case some are blocked off
  2. Any data collected via the Android app can be sent via email to a dedicated tallying center address. To ensure that only data from the app is received and not changed in transit, encryption is used.

The inspiration came from a quote by Ghandi “Be the change you wish to see in the world”, disproving the myth that there is no local capability to design and implement such solutions and most of all that such solutions have to be complex.

Looking forward to hearing your thoughts and suggestions…

12 May

Opinion: Microsoft Demise being Overrated but they are still a Major Player

Posted by ssmusoke in database, technology, windows. Tagged: .

This post was intially a comment on this Mashable post Why Microsoft Is Being Left in the Dust but I figured it was too long and needed its own post

The demise of Microsoft has been predicted continuously over the last 20 years however what most commentators forget is that Microsoft innovates best when its the under-dog:

a) Browsers – beat Netscape to a pulp and was the browser kng for the next 10 year do I hear IE 6, and today after steady decline IE has 50% market share and holding steady, watch out for a rise with IE 10

b) Desktop – anybody remember OS2

c) Office Productivity – Wordperfect, Lotus 1-2-3 anyone

d) Exchange – Novel Groupware

e) Networking/Active Directory – Novel Netware, hey Windows NT, how many people use Windows boxes for Active Directory and File and Print against Samba

f) Xbox – do I hear Nintendo DS/PS3 is just picking up but Sony is suffering

g) Databases – Watchout Oracle/DB2 SQL server is deep within the departments and getting many enterprise features

h) Antivirus – McAfee/Norton/Kaspersky watch out for the free MS Security Essentials I know I have never looked back

i) Corporate Intranets – Documentum watch out for Sharepoint

j) Open Source – top 10 contributer to Linux, being a developer PHP/MySQL support for windows excellent

k) Programming Languages – .NET has caught up, seems like each and every NGO/UN department is running licensed Windows/IIS/SQL Server/Sharepoint/Exchange

l) Android – who has made the most money from licensing patents, Samsung/HTC/Barnes and Noble/Motorolla are all paying

m) Mobile – they may be late to this game, but PCs are here to stay and compliment smartphones/tablets, so there is no danger there, Linux is not yet mature enough for the desktop and Mac OSX not available

n) Research Labs – apart from IBM which uses research as a competitive edge, HP closed down their, Google are still in the game, the only other company in this big time seems to be Redmond

o) Healthcare – with those US automation dollars flowing down

p) Enterprise Applications – they may have faltered but this market is growing with Navison and Dynamics

q) Channel – who else is bigger and better at harnessing this resource, these are an extension of the sales force

r) Development tools – I do not use them but they are the envy of many a developer

Please share your thougts and opinions

« Older Entries