Theo Todman's Web Page
For Text Colour-conventions (at end of page): Click HereStatus Reports - Status: Web-Tools (2019 - June)
Rationale for this ProjectNote last updated: 14/07/2019 20:29:46
- This Project was alluded to briefly in a footnote on research methodology in my original Research Proposal under the head Research - Internet Technology. When last at Birkbeck, I wrote a more extensive paper defending the Project and describing its rationale. Now that my PhD is in suspense, I have decided to take this Project further. There’s a lot to do: still quite a few items on the “wish list”. It is fairly critical as an enabler for my research, so I need to get a move on as I want it all out of the way before I re-start4 formal research.
- For documentation on my website (currently password protected) follow the links below:-
- Functional5 Documentation.
- Technical7 Documentation.
- Other Websites
- Sometime around 2005, I created a website for Dr. Sophie Botros (Sophie Botros: Live Site (http://www.sophiebotros.com/)), one of my supervisors at Birkbeck, but we then lost touch and it got maintained (very badly) by some desktop support outfit. In 15Q2 I took it back on again and spruced it up a bit, and maintain it periodically, though it still requires more work.
- I created and / or ran a multitude of bridge websites, but as of January 2018 I have either handed them over or mothballed them9:-
- I established the Hutton Bridge Club (http://www.bridgewebs.com/hutton.html) Website in 11Q4 using the standard Bridgewebs service, but with a couple of competitions using my own routines. This was handed over in 15Q3.
- In 15Q1, I took over the support and development of the Essex Contract Bridge Association (http://www.bridgewebs.com/essex/) (ECBA) website, which also uses Bridgewebs, but is very much larger. I wrote a lot of code10 to make this job less tedious. The site was handed over in 17Q4.
- For several years, I collected data11 on bridge activity in the Billericay / Brentwood area (initially needed for a project to set up a new consolidated club) – by “scraping” data off web pages, consolidating it into a database and modelling it in various ways.
- I used this data to generate websites with a multitude of ladders for small clubs (Essex Bridge Results (http://www.theotodman.com/Bridge/index.htm)). These are now mothballed.
- I created and maintained a new website for the First Class Bridge (http://www.bridgewebs.com/firstclassbridge/) Academy, giving it “small clubs” ladders as these were easy to maintain with little intervention.
- In 16Q3 I revived the Mountnessing Bridge Club Archive (http://mountnessing.theotodman.com/index.shtml) website.
- I’ve created and continue to maintain a small website for a music group Julie and I attend – the Enigma Ensemble (http://www.enigmaensemble.co.uk/).
- I created a website for displaying the textual and grammatical analyses and appendices of Pete’s PhD on the Acts of the Apostles. It exists in two versions: Acts: Live Site (http://www.acts-research.website/) and Acts: Test Site (http://www.theotodman.com/Petes_PhD/index.htm).
- I did consider returning to work part-time as an MS Access/Excel developer, with a spin-off into website generation, but have done nothing about it so far.
Summary of Progress during April - June 2019
Website (Total Hours = 72)
- I spent 73 hours in 19Q2 on this Project, or related work (332.25 hours YTD, where for "YTD" - Year to Date - I mean the (academic) year that commenced in October 2018). That's 94.6% of the planned effort (106.6% YTD). Overall, 12.1% of my Project effort in the Quarter was directed towards this project (making 18.2% YTD) - as against 12% planned (23.2% YTD).
- Again, I made no progress at all on the courses put out by Harvard
→ "Harvard University - CS50 Introduction to Computer Science", and
I still intend to make a serious attempt to complete the former sometime (I will need to re-start). However, I have decided to precede it with the latter which is more immediately relevant. Unfortunately, I’ve no time available to pursue either course at the moment.
- Sophie Botros has let me know that she’s having her website professionally redeveloped by Bookswarm (https://bookswarm.co.uk/).
- I’ve stuck to the greatly reduced “budget” of only 6 hours a week, and approximately achieved it this time.
- Completed items included:-
- Own Website:
- Added Jump Table to sections of Authors Pages
- Created 'Chess' Home Page
- Created as a main menu item and Note
- Created consolidated results pages
- Added games & links to pgn4web
- Corrected Quarterly Reporting for uneven Plan (Holidays)
- Other Websites:
- Full details for 19Q2 are given below:-
- Website - Bridge Development
- Website - Bridge Maintenance (Total Hours = 3.75)
- Bridge - ECBA Website - Assistance to Petra (OneDrive) (0.25 hours)
- Bridge - Website - Results Database Maintenance (3.5 hours)
→ See "Admin - Bridge - Admin" (3.75 hours)
- Website - Development (Total Hours = 36.5)
- Review Webalizer Methodology & Consolidate Stats (0.5 hours)
- Website - 'Chess' as a main menu-item to Website (0.75 hours)
- Website - Chess Results & Games Pages - Create & Maintain (6.25 hours)
- Website - Generator - Add a pick-list to my Priority Task List (Functor_10) (0.5 hours)
- Website - Generator - Add Jump Table to sections of Authors Pages (3.5 hours)
- Website - Generator - Add 'Statistics' Comment to Book/Paper Summaries (1.5 hours)
- Website - Generator - Automatic Key-word Hyperlinking to Notes Project: Bug - Authors not being updated (0.25 hours)
- Website - Generator - Automatic Key-word Hyperlinking to Notes Project: Remove 'Methodology' to linked Note (0.25 hours)
- Website - Generator - Chess Results & Games Pages - Add pgn4web (6 hours)
- Website - Generator - Chess Results & Games Pages - Create & Maintain (4.25 hours)
- Website - Generator - Correct Quarterly Reporting for uneven Plan (Holidays) (3.5 hours)
- Website - Generator - Document Functional Overview (1.25 hours)
- Website - Generator - Document Printable Notes (4 hours)
- Website - Generator - Fix Bug - Error 62 (Input Past EOF) for degenerate files (1.25 hours)
- Website - Generator - Fixed obscure bug in CreateAuthorsWebPages for new Authors (0.5 hours)
- Website - Generator - Improve automatic WebRefs Checking (0.75 hours)
- Website - Generator - Periodic Full Regeneration - Add debug-print of stage timestamps (1.5 hours)
→ See "Software Development - Website - Development" (36.5 hours)
- Website - Education
- Website - Infrastructure (Total Hours = 14.75)
- Website - Maintenance (Total Hours = 16)
- 19Q1 Status Reports (2.25 hours)
- Website - Generator - WebRefs - Manual / Automatic URL Checks (2.5 hours)
- Website - Maintain Text of Family Page (2.5 hours)
- Website - Maintain Text of my Personal Page (2.25 hours)
- Website - Periodic Full Regeneration (4.5 hours)
- Website - Run Web Spider (1.25 hours)
- Website - ZoomSearch database refresh (0.75 hours)
→ See "Admin - Website - Admin & Maintenance" (16 hours)
Plans for the Near Future
I’ve had to maintain the planned weekly effort on this project at the much reduced 6 hours in order to allow for other commitments. While there’s still a lot on my development “priority list”, my hope for the current academic year was to focus on updating my technical competence. However, I can’t see how to fit it in. The Plan below is taken automatically from the Priority 1 items on my Development Log, as published in my Outstanding Developments Report.
- Own Website: Priority 1 Items By Category:-
- Complete XRef-re-engineering project:-
- Ensure all links and link-pages use the new XRef table, and pension off the old tables.
- Check all link-types still work and fix any errors.
- Complete the auto-triggering of regeneration of “associated” link pages.
- Fix update bug in Convert_Webrefs.
- Fix Bug whereby PaperSummary pages seem to have “Works-” and “Books/Papers-” Citings that refer to the same link-pages.
- Review effectiveness of hyperlinking method in the light of PhD and Philosophy of Religion experience.
- Where possible, use ID rather than NAME for in-page hyperlinks
- As revealed by Spider: Author pages not regenerated when name corrected, leading to Sundry broken Links from other pages. 72 items.
- As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items.
- Review architecture to improve performance; Need to document first
- Investigate whether multiple Subject/Topic/Subtopic usage leads anywhere (ie. are just the first (of 3) actually used). Fix anything amiss.
- Provide Functional Documentation for Website Generator (using Notes)
- "Sitepoint (Learnable) - Sitepoint Learnable Web Development Courses": Plan what to do (and with the eBooks in my possession).
- "Harvard University - CS50 Introduction to Computer Science": Re-start, or maybe just "audit".
- Bitcoin & Blockchain: Understand better
- Bootstrap: Investigate.
- Read "PC Pro - Computing in the Real World".
- iCloud: Re-install & solve 'The upload folder for iCloud Photos is missing' problem
- Purchase & Commission new Lenovo Laptop
- Add "Note Alternates" to Note pages.
- Allow the option to concatenate Notes in the Printed version (ie. linearly embed them essay-style), rather than treating the hyperlinks as footnotes – but still keep the hyperlink & cross-referencing in place.
- For use as "disclaimers" - eg. for "Plug Notes".
- For Thesis / essays: the difficulty here is the need for linking passages to make the text run smoothly.
- As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
- As revealed by Spider: Blog Achived Notes - sundry links to Printable Notes. Live Note OK. 110 items. 100 items outstanding.
- As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
- Investigate Note_Links: Section references seem to be incorrect
- Printable Notes: fix the bug whereby the “private” flag is round the wrong way.
- Suppress the publication of the Printable versions of Temp Notes
- Develop software & procedure to make adding more content to the photos pages easier to undertake.
- Timeline software: Add photos for Holidays
- As revealed by Spider: Sundry uncategorised. Refs failing. 17 items.
- Determine why Recalculation & Changed Book/Papers produce unneeded regeneration.
- Full Website Regeneration is now taking 35 hours. Investigate why so, and improve performance!
- Analyse the results of the data collection exercise and design a plan of campaign to fix broken Internal links and prevent recurrence.
- Correct the code so the problems discovered by the Spider don’t recur.
- Delete 'orphan pages' that are never linked to, ie. Use the Spider to prune redundant pages17 automatically where possible.
- Fix the historical data where errors are uncovered by the Spider. An easier task now the site has a full-regen function.
- The Spider was generating WebRefs. Procedurally, this ought not to have been possible.
I've re-opened the case!
- The major problem turned out to be because unprocessed18 URLs got added to the end of the last WebLinks_Tester_Brief page, which then got Spidered. I've stopped this happening, so hopefully the problem will not recur. The fix was made in 18Q2.
- However, 4 other creations appeared - dated 18/05/18 - from the run of 10/07/18. The creation date was from the previous spider run, but the IDs show that they were produced in the latest run.
- Look into Sistrix Smart19. Errors and warnings itemised are:-
- Duplicate content: seems to be variants on theotodman.com
- Title Tags: Empty, too long, identical
- Page Not Found
- Filesize in excess of 1Mb
- Meta-Description: Empty
- Few words on Page
- H1: Not used, used multiple times per page, identical across pages
- Pictures: Alt attribute missing
- As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 116 items.
- As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 16 items.
- Documentation & Bug-fixes: Phase 2
- Re-document the procedures in the light of recent changes.
- Resolve issues generated / revealed by the spider.
- Investigate - and fix where possible - broken links.
- Reformat WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm
- Clarify 'truncated': Display, not link
- Allow more space for 'link returned', 'issue' and 'display text'
- The 'As Above" lines waste space. Only for Notes Archive? Consolidate onto single second line.
- Reformat WebLinks_Tester_Brief: Allow more space for 'link returned', 'issue' and 'display text'
- Other Websites: Priority 1 Items By Category:-
- Create "Small Sites" database for Alaric (if chased)
- Termly updates to the Enigma Ensemble (http://www.enigmaensemble.co.uk/) Website.
- Periodic updates to Sophie Botros: Live Site (http://www.sophiebotros.com/)
Summary of Progress to Date
This is hived off to various separate documents, which have now been harmonising and / or consolidated:-
- Summary of Progress to Date.
- Outstanding Developments,
- Functional Documentation,
- A summary of time expended across the years developing my website23 is at "Software Development - Website - Development".
- Well, I’ve missed the boat, so will need to continue with both projects in parallel.
- This was always likely to be necessary, as new features will always arise in use. It’s a prototype methodology, after all.
- This is very tedious to produce and consequently is both incomplete and out of date.
- This is much more fun, as it’s a purely technical task.
- I’ve written a vastly-improved general-purpose technical documenter for MS Access.
- It’s a shame to abandon the “mini websites” with all their ladders, as it’s rather well done.
- However, I couldn’t waste time on these after I’d abandoned bridge.
- In particular, for the ECBA “Victor Ludorum” competition.
- I cannot hand any of this code over, so the tedium will return, though not to me!
- I had agreed to share this data sometime early in 2018
- But will wait until asked again, as I doubt it’ll be of any real use to anyone.
- Note that Backup_Prune_Ctrl deletes (relevant) pages that weren't regenerated in the last full site-regen, but this isn't the same thing.
- These are URLs that were used in web pages but hadn't yet been converted to the +WnnnW+ format, so appeared at the end with no WebRef ID.
- See Sistrix (https://www.sistrix.com/smart/)
- This used to be called Optimizr, see Optimizr (http://www.optimizr.com/) (which now auto-forwards to Sistrix).
- A quick look doesn’t show it to be an obvious scam, but I need to double-check.
- An unsolicited analysis of my site turned up monthly from Optimizr from January 2015 to October 2017, listing a large number of “problems” that I think I know about, but which are in the queue to address.
- It restarted in February 2018, under the Sistrix name (this seems to have been associated with Optimizr since November 2015).
- The free version of this software is restricted to 1,000 pages, which is a very small proportion of my Site, though I may be able to point it to difference base-URLs.
- But I do need to address the problems validly itemised, and a sub-set is still useful.
- As distinct from developing other peoples’ websites – time which is also recorded against this project, but not against this task.
Text Colour Conventions
- Black: Printable Text by me; © Theo Todman, 2019
- Blue: Text by me; © Theo Todman, 2019
Return to Home page
Timestamp: 16/07/2019 17:25:57. Comments to email@example.com.