During my professional career I have moved through the ranks of IT and I try to stay in touch with hands-on work on software design and programming although for the last 25 years or so my roles have been in Program and Project Management
Recently I attended a master class on Artificial Intelligence and Machine Learning at the University of Technology Sydney (UTS), and that opened my apetite for getting into the details of that kind of implementations using Python Laguage. This also took me into the Web development techniques which I have used to build this site.
One key motivation was that, eventually, I will be requested to manage a troubled project trying to implement this stuff…
Another motivation is the fact that I am a nerd at heart… I am (or used to be) a C++ programmer, I do quite a bit of VBA coding and other hands-on implementations to assist me in the management of my projects and programs.
I decided that my studies would be in my own discretionary time (i.e. no time off from work) and at the lowest possible cost. Also, I decided that I would keep a record of my studies.
Over the last few months I have been exploring different sources of information and I have worked in a number of little personal projects more or less concurrently.
I would like to share with you my experiences and progress.
The following table shows the main streams of work undertaken for my studies. At the moment, the main focus is the Personal Blog Application which I intend to improve and enrich with the work coming from the other streams.
|
ID |
Project / Stream |
Description |
Main Python tools/modules |
|
CTX01 |
Automating stuff |
Automating stuff such as WebPage scrapping, merging spreadsheets, etc. |
· Python development environment set-up · Beautiful soup · Anaconda · PIP |
|
CTX02 |
Schedule Management Software |
Re-development and improvement of software that I have written to assist myself and colleagues in the creation, integration and reporting on schedules in complex projects and programs |
· Win32com.Client · Pitz · datetime, dateutil.parser · SQLite |
|
CTX03 |
Personal Blog Application |
Development of fully functional and secured web application using Flask. The application includes authentication and password encryption Deployed in production in December 2019 on a cloud Linux server, this included shopping/selecting the hosting services provider Setting up NGINX, gunicorn and Supervisor as well as securing the site by moving to https. In Jan-2020 this is my flag-ship project. The original purpose was to go through the technical issues and challenges. Past that stage, this project is now to improve content and presentation and make it a marketing/branding tool. |
· Flask · WTF · Bootstrap · SQLite · Blueprint · itsdangerous |
|
CTX04 |
Data Maps |
For this I am using libraries that facilitate the extraction and manipulation of data combined with the ability to present this data in charts and geographical maps, similar to Google maps. Since I was born in Chile and I live in Australia, I have explored the building of maps with a choropleth data visualization of Chilean migration to Australia. For that I have downloaded data from the Australian Bureau of Statistics and used data analytics techniques/modules |
· GeoJSON files · Folium · Choropleth maps · Pandas · Numpy · Webbrouser |
|
CTX05 |
Machine Learning |
After attending a master class on this topic in the UTS, I started from the very basic concepts and built the knowledge almost from scratch. My approach was to get very familiar with the concepts and theory before attempting any coding. Since I have a good base on Linear Algebra and Calculus, I am now very familiar with algorithms for: · Linear Regression · Support Vector Machine · Random Forrest Classifier · Neural Networks. By now, I understand and I am able to apply the backpropagation algorithm for the optimisation of neural networks. I have written a few mini routines for hand-written digits recognition, voice recognition and my plan is to partner with colleagues and/or organisations to apply these concepts in specific client’s situations. |
· Pickle · Gzip · mnist_loader · Jason · Random · matplotlib · Theano · Seaborn · pylab · SciKit Learn libraries for: · Image processing (e.g. Pillow) · speech_recognition |
As I indicated above, there are tons of these things out there. Each project uses a starting set. This set grows rapidly as your project increases in sophistication… following is the ‘backbone’ list that I am using:
|
Module |
Description |
|
Flask |
Web development Framework |
|
WTF |
Forms for web development |
|
matplotlib |
Charts and graphics |
|
Folium |
Maps creation (similar to google maps) |
|
Pandas |
Data analytics, data imports, matrices ops, etc |
|
Numpy |
Mathematical functions, matrices, random generators |
|
SQLite |
Data base management |
|
SciKit |
For machine learning |
|
Win32com.Client |
To work with MS applications such as MSProject |
|
Pitz, datetime, dateutil.parser |
To work with dates/times… |
|
Openpyxl |
For Excel |
|
itsdangerous |
Creation of Tokens for password resetting |
|
smtplib |
For sending emails |
I have bought a couple of books to help with the learning. These include:
I have found lots of good material (also some pretty bad ones). I find the following authors really good:
Many more but these guys are really good! Gurus.
Tutorials are free but we can make a donation if we have the means. (I joined patreon and have bought some of their books…)
I provide a list of recommended sites at the end of this page.