Theo Todman's Web Page

For Text Colour-conventions (at end of page): Click Here

Status Reports - Status: Web-Tools (2019 - March)

Rationale for this Project


Summary of Progress during January - March 2019
  1. I spent 179 hours in 19Q1 on this Project, or related work (258.5 hours YTD, where for "YTD" - Year to Date - I mean the (academic) year that commenced in October 2018). That's 234.3% of the planned effort (111% YTD). Overall, 28.7% of my Project effort in the Quarter was directed towards this project (making 21.2% YTD) - as against 11.1% planned (16.7% YTD).
  2. Again, I made no progress at all on the courses put out by Harvard
    → "Harvard University - CS50 Introduction to Computer Science", and
    → "Harvard University - CS50W Web Programming with Python and JavaScript".
    I still intend to make a serious attempt to complete the former sometime (I will need to re-start). However, I have decided to precede it with the latter which is more immediately relevant. Unfortunately, I’ve no time available to pursue either course at the moment.
  3. I’d greatly reduced my “budget” to only 6 hours a week, but more than doubled this.
  4. The main reason for this was the urgent need to create Timeline software and populate the pages with photos of the developments and repairs to Coxes Farm.
  5. I also created a “Hits Page” – based on the statistic provided by Webalizer (http://www.webalizer.org/). This requires analysis, to determine whether my site is a benefit to, or a blight on, the philosophical community.
  6. Completed items included:-
    • Books/Papers
      1. Removed Blackberry processing and associated pages as no longer needed with fast broadband.
    • Photos
      1. Timeline software: Added photos of Coxes Farm
      2. Created Timeline software
      3. Created Photo-logging usage via Images table.
    • Technology
      1. Investigated the Hit Stats provided by Webalizer and produced web-pages to show the most active pages.
  7. Full details for 19Q1 are given below:-
Website (Total Hours = 179)
  1. Website - Bridge Development
  2. Website - Development (Total Hours = 160.25)
    • Reference_Author: Re-format 'Not found' Author Links (0.25 hours)
    • Add 'Chess' as a main menu-item to Website (0.5 hours)
    • Create Timeline software (49 hours)
    • Remove Blackberry Pages & Processing (1 hour)
    • Review Webalizer Methodology & Consolidate Stats (13 hours)
    • Todman (Theo) - Tottering Towers & Listing Buildings: Add / annotate photos of Coxes Farm to Timeline (79.25 hours)
    • Website - Generator - Document Referencing functions (plus sundry referencing fixes) (4 hours)
    • Website - Generator - Document Timeline functions (4 hours)
    • Website - Generator - Image Link Recording (Images) (4 hours)
    • Website - Generator - Password Protection - Review (0.25 hours)
    • Website - Generator - Refine totals in Functor_08 (2 hours)
    • Website - Generator - Run and Refine File Pruning process (0.5 hours)
    • Website - Investigating https / ssl (0.75 hours)
    • Website - Periodic Full Regeneration - Add debug-print of stage timestamps (1 hour)
    • Website - Update 'Photos' Page to link to Coxes Farm Photo Pages (0.75 hours)
      → See "Software Development - Website - Development" (17.25 hours)
  3. Website - Education (Total Hours = 1.25)
  4. Website - Infrastructure (Total Hours = 6)
    • Install Kaspersky on own & Julie's laptops (1 hour)
    • Investigation: PCRisk - Your Account Was Hacked Email Scam (https://www.pcrisk.com/removal-guides/14469-your-account-was-hacked-email-scam) (0.25 hours)
    • Microsoft Windows 10 / MS Office - Releases, Bugs & Periodic Re-boots (3 hours)
    • Nat's monitor - order, chase, complain! (0.5 hours)
    • PC Backups / OneDrive (0.75 hours)
    • Renewal of BT Internet connection (0.25 hours)
    • Spurious cancellation of BT Internet connection - reversed! (0.25 hours)
      → See "Software Development - Website - Development" (6 hours)
  5. Website - Maintenance (Total Hours = 10.5)
    • 18Q4 Status Reports (1.25 hours)
    • Website - Generator - WebRefs - Manual URL Checks (0.25 hours)
    • Website - Periodic Full Regeneration (6.5 hours)
    • Website - Run Web Spider (1.5 hours)
    • Website - ZoomSearch database refresh (1 hour)
      → See "Software Development - Website - Development" (9.25 hours)
Website Others
Plans for the Near Future

I’ve had to maintain the planned weekly effort on this project at the much reduced 6 hours in order to allow for other commitments. While there’s still a lot on my development “priority list”, my hope for the current academic year was to focus on updating my technical competence. However, I can’t see how to fit it in. The Plan below is taken automatically from the Priority 1 items on my Development Log, as published in my Outstanding Developments Report.
  1. Own Website: Priority 1 Items By Category:-
    • Architecture
      1. Complete XRef-re-engineering project:-
        1. Ensure all links and link-pages use the new XRef table, and pension off the old tables.
        2. Check all link-types still work and fix any errors.
        3. Complete the auto-triggering of regeneration of “associated” link pages.
        4. Fix update bug in Convert_Webrefs.
        5. Fix Bug whereby PaperSummary pages seem to have “Works-” and “Books/Papers-” Citings that refer to the same link-pages.
      2. Review effectiveness of hyperlinking method in the light of PhD and Philosophy of Religion experience.
      3. Where possible, use ID rather than NAME for in-page hyperlinks
    • Authors
      1. As revealed by Spider: Author pages not regenerated when name corrected, leading to Sundry broken Links from other pages. 72 items.
      2. As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items.
    • Backups
      1. Review architecture to improve performance; Need to document first
    • Books/Papers
      1. Investigate whether multiple Subject/Topic/Subtopic usage leads anywhere (ie. are just the first (of 3) actually used). Fix anything amiss.
    • Documenter
      1. Provide Functional Documentation for Website Generator (using Notes)
    • Education
      1. "Sitepoint (Learnable) - Sitepoint Learnable Web Development Courses": Plan what to do (and with the eBooks in my possession).
      2. "Harvard University - CS50 Introduction to Computer Science": Re-start, or maybe just "audit".
      3. "Harvard University - CS50W Web Programming with Python and JavaScript": either complete it in 2019, or check it'll still run in 2020.
      4. Bitcoin & Blockchain: Understand better
      5. Bootstrap: Investigate.
      6. Read "PC Pro - Computing in the Real World".
    • Infrastructure
      1. iCloud: Re-install & solve 'The upload folder for iCloud Photos is missing' problem
    • Notes
      1. Add "Note Alternates" to Note pages.
      2. Allow the option to concatenate Notes in the Printed version (ie. linearly embed them essay-style), rather than treating the hyperlinks as footnotes – but still keep the hyperlink & cross-referencing in place.
        1. For use as "disclaimers" - eg. for "Plug Notes".
        2. For Thesis / essays: the difficulty here is the need for linking passages to make the text run smoothly.
      3. As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
      4. As revealed by Spider: Blog Achived Notes - sundry links to Printable Notes. Live Note OK. 110 items. 100 items outstanding.
      5. As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
      6. Investigate Note_Links: Section references seem to be incorrect
      7. Printable Notes: fix the bug whereby the “private” flag is round the wrong way.
      8. Suppress the publication of the Printable versions of Temp Notes
    • Photos
      1. Develop software & procedure to make adding more content to the photos pages easier to undertake.
      2. Timeline software: Add photos for Holidays
    • Process
      1. As revealed by Spider: Sundry uncategorised. Refs failing. 17 items.
      2. Determine why Recalculation & Changed Book/Papers produce unneeded regeneration.
      3. Full Website Regeneration is now taking 35 hours. Investigate why so, and improve performance!
    • Spider
      1. Analyse the results of the data collection exercise and design a plan of campaign to fix broken Internal links and prevent recurrence.
        1. Correct the code so the problems discovered by the Spider don’t recur.
        2. Delete 'orphan pages' that are never linked to, ie. Use the Spider to prune redundant pages15 automatically where possible.
        3. Fix the historical data where errors are uncovered by the Spider. An easier task now the site has a full-regen function.
      2. The Spider was generating WebRefs. Procedurally, this ought not to have been possible.
        1. The major problem turned out to be because unprocessed16 URLs got added to the end of the last WebLinks_Tester_Brief page, which then got Spidered. I've stopped this happening, so hopefully the problem will not recur. The fix was made in 18Q2.
        2. However, 4 other creations appeared - dated 18/05/18 - from the run of 10/07/18. The creation date was from the previous spider run, but the IDs show that they were produced in the latest run.
        I've re-opened the case!
    • Technology
      1. Look into Sistrix Smart17. Errors and warnings itemised are:-
        1. Duplicate content: seems to be theotodman.com and Link - Defunct
        2. Title Tags: Empty, too long, identical
        3. Page Not Found
        4. Filesize in excess of 1Mb
        5. Meta-Description: Empty
        6. Few words on Page
        7. H1: Not used, used multiple times per page, identical across pages
        8. Pictures: Alt attribute missing
    • WebRefs
      1. As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 116 items.
      2. As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 16 items.
      3. Documentation & Bug-fixes: Phase 2
        1. Re-document the procedures in the light of recent changes.
        2. Resolve issues generated / revealed by the spider.
        3. Investigate - and fix where possible - broken links.
      4. Reformat WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm
        1. Clarify 'truncated': Display, not link
        2. Allow more space for 'link returned', 'issue' and 'display text'
        3. The 'As Above" lines waste space. Only for Notes Archive? Consolidate onto single second line.
      5. Reformat WebLinks_Tester_Brief: Allow more space for 'link returned', 'issue' and 'display text'
  2. Other Websites: Priority 1 Items By Category:-
    • Bridge
      1. Create "Small Sites" database for Alaric (if chased)
    • Enigma
      1. Termly updates to the Enigma Ensemble (http://www.enigmaensemble.co.uk/) Website.
    • Sophie
      1. Periodic updates to Sophie Botros: Live Site (http://www.sophiebotros.com/)

Summary of Progress to Date

This is hived off to various separate documents, which have now been harmonising and / or consolidated:-
  1. Summary of Progress to Date.
  2. Outstanding Developments,
  3. Functional Documentation,
  4. A summary of time expended across the years developing my website21 is at "Software Development - Website - Development".



In-Page Footnotes

Footnote 4:
  • Well, I’ve missed the boat, so will need to continue with both projects in parallel.
  • This was always likely to be necessary, as new features will always arise in use. It’s a prototype methodology, after all.
Footnote 5:
  • This is very tedious to produce and consequently is both incomplete and out of date.
Footnote 7:
  • This is much more fun, as it’s a purely technical task.
  • I’ve written a vastly-improved general-purpose technical documenter for MS Access.
Footnote 9:
  • It’s a shame to abandon the “mini websites” with all their ladders, as it’s rather well done.
  • However, I couldn’t waste time on these after I’d abandoned bridge.
Footnote 10:
  • In particular, for the ECBA “Victor Ludorum” competition.
  • I cannot hand any of this code over, so the tedium will return, though not to me!
Footnote 11:
  • I had agreed to share this data sometime early in 2018
  • But will wait until asked again, as I doubt it’ll be of any real use to anyone.
Footnote 15:
  • Note that Backup_Prune_Ctrl deletes (relevant) pages that weren't regenerated in the last full site-regen, but this isn't the same thing.
Footnote 16:
  • These are URLs that were used in web pages but hadn't yet been converted to the +WnnnW+ format, so appeared at the end with no WebRef ID.
Footnote 17:
  • See Sistrix (https://www.sistrix.com/smart/)
  • This used to be called Optimizr, see Optimizr (http://www.optimizr.com/) (which now auto-forwards to Sistrix).
  • A quick look doesn’t show it to be an obvious scam, but I need to double-check.
  • An unsolicited analysis of my site turned up monthly from Optimizr from January 2015 to October 2017, listing a large number of “problems” that I think I know about, but which are in the queue to address.
  • It restarted in February 2018, under the Sistrix name (this seems to have been associated with Optimizr since November 2015).
  • The free version of this software is restricted to 1,000 pages, which is a very small proportion of my Site, though I may be able to point it to difference base-URLs.
  • But I do need to address the problems validly itemised, and a sub-set is still useful.
Footnote 21:
  • As distinct from developing other peoples’ websites – time which is also recorded against this project, but not against this task.


Note last updated: 05/04/2019 10:36:29



Text Colour Conventions

  1. Black: Printable Text by me; © Theo Todman, 2019
  2. Blue: Text by me; © Theo Todman, 2019


Return to Home page
Timestamp: 07/04/2019 09:53:29. Comments to theo@theotodman.com.