Theo Todman's Web Page - Notes Pages


Status Reports

Status: Web-Tools (2019 - December)

(Text as at 19/01/2020 23:41:17)

(For earlier versions of this Note, see the table at the end)


Rationale for this Project


Summary of Progress during October - December 2019
  1. I spent 94.75 hours in 19Q4 on this Project, or related work (94.75 hours YTD, where for "YTD" - Year to Date - I mean the (academic) year that commenced in October 2019). That's 104.6% of the planned effort (104.6% YTD). Overall, 14% of my Project effort in the Quarter was directed towards this project (making 14% YTD) - as against 11.7% planned (11.7% YTD).
  2. Completed items included:-
    1. Own Website:
      • Animadversions
        1. Create Aeon_Files table & code to generate Aeon Page12 (to simplify processing and get round the 64k character limit).
      • Authors
        1. Improve author-citations process:-
          1. Show author-citation counts on Author Summary pages.
          2. Only have an author-citation link on Authors pages if the author is cited!
          3. Import Authors from Aeon_Files table.
      • Notes
        1. Create procedures (manual and via Functor_17) to add Audio Files to Notes
      • Process
          1. Full Website Regeneration failed in Notes Regen because the first Private Note (13313) couldn't be copied from the Transfer directory as the private note hadn't been created there by then.
          2. It should have been, from looking at the code.
          3. Also, the private note itself (116314) is not regenerated until later.
          4. In the start-December 2019 run, no such problem occurred. Maybe the problem was caused by copied between laptops. So, I've flagged it as complete for now, pending a recurrence.
        1. Fix Functors to enable previous quarter's Status Reports after Time period roll-over
    2. Other Websites:
      • Nothing to Report.
  3. Full details for 19Q4 are given below15:-
Website (Total Hours = 94.25)
  1. Website - Bridge Development (Total Hours = 5.25)
    • Bridge - Website - Auto-reconciliation vs EBU (5 hours)
    • Bridge - Website - Development - Synchronise Documentator with main website generator (0.25 hours)
      → See "Admin - Bridge - Admin" (5.25 hours)
  2. Website - Bridge Maintenance
  3. Website - Development (Total Hours = 55.5)
    • Review Webalizer Methodology & Consolidate Stats (0.5 hours)
      → See "Software Development - Website - Development" (0.5 hours)
    • Todman (Theo) - Tottering Towers & Listing Buildings: Add / annotate photos of Coxes Farm to Timeline (0.25 hours)
      → See "Todman (Theo) - Tottering Towers & Listing Buildings" (0.25 hours)
    • Website - Backup - Copying errors (0.25 hours)
    • Website - Backup - Count discrepancies (0.5 hours)
    • Website - Generator - Add Audio Files to Notes (3.25 hours)
    • Website - Generator - Author Citation list improvements (2.25 hours)
    • Website - Generator - Automate Aeon Page output (28.5 hours)
    • Website - Generator - Compact / Repair re-open '2Gb' alleged problem - increase MaxLocksPerFile (0.5 hours)
    • Website - Generator - Create Functor to facilitate addition of Audio Files to Music Page (4.75 hours)
    • Website - Generator - Develop oboe-practice report (4.5 hours)
    • Website - Generator - Fix bug in Plan versus Actual Effort Summary - Effort (Current Quarter & YTD) - YTD Actual % v Plan (1 hour)
    • Website - Generator - Fix Functors to enable previous quarter's Status Reports after Time period roll-over (5.75 hours)
    • Website - Generator - Fixes re Broken Links revealed by Spider (0.75 hours)
    • Website - Homepage - Review & Re-balance (0.5 hours)
    • Website - Periodic Full Regeneration - Error copying Private Note (1.5 hours)
    • Website - ZoomSearch database refresh - Improve processing (0.75 hours)
      → See "Software Development - Website - Development" (54.75 hours)
  4. Website - Education (Total Hours = 2.5)
  5. Website - Infrastructure (Total Hours = 10)
    • Chrome bookmark de-duplication (0.75 hours)
    • EE Broadband - Renew & Install new Router (0.25 hours)
    • iCloud for Windows Installation (0.75 hours)
    • Microsoft Windows 10 / MS Office - Releases, Bugs & Periodic Re-boots (0.75 hours)
    • PC Backups / OneDrive (1.5 hours)
    • Printer - New Drum & Toner (0.5 hours)
    • Printer - Re-order Toner & Drum (0.25 hours)
    • Sky Q Order & Installation + new TV (3.5 hours)
      → See "Admin - Website - Admin & Maintenance" (8.25 hours)
    • XMas Newsletter - photos & formatting (1.75 hours)
      → See "Software Development - Website - Development" (1.75 hours)
  6. Website - Maintenance (Total Hours = 14.5)
    • 19Q3 Status Reports (3 hours)
    • 19Q4 Status Reports (0.25 hours)
      → See "Admin - Website - Admin & Maintenance" (3.25 hours)
    • Website - Generator - Chess Results & Games Pages - Create & Maintain (2.5 hours)
      → See "Software Development - Website - Development" (2.5 hours)
    • Website - Generator - WebRefs - Manual / Automatic URL Checks & Fixes (0.5 hours)
    • Website - Maintain 'Websites supported' Page (0.25 hours)
    • Website - Periodic Full Regeneration (5 hours)
    • Website - Run Web Spider (1.25 hours)
    • Website - ZoomSearch database refresh (1.75 hours)
      → See "Admin - Website - Admin & Maintenance" (8.75 hours)
Website Others
Plans for the Near Future

The Plan below is taken automatically from the Priority 1 items on my Development Log, as published in my Outstanding Developments16 Report. I’ve maintained the weekly allocation at 7 hours.
  1. Own Website: Priority 1 Items By Category:-
    • Architecture
      1. Complete XRef-re-engineering project:-
        1. Ensure all links and link-pages use the new XRef table, and pension off the old tables.
        2. Check all link-types still work and fix any errors.
        3. Complete the auto-triggering of regeneration of “associated” link pages.
        4. Fix update bug in Convert_Webrefs.
        5. Fix Bug whereby PaperSummary pages seem to have “Works-” and “Books/Papers-” Citings that refer to the same link-pages.
      2. Review effectiveness of hyperlinking method in the light of PhD and Philosophy of Religion experience.
      3. Where possible, use ID rather than NAME for in-page hyperlinks
    • Authors
      1. As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items.
      2. Completely re-engineer Authors processing to use IDs rather than Names:-
        1. Authors' pages to be identified by Author_nnnnn, where nnnnn is the (numeric) Author_ID.
        2. Directory-structure in thousands: Author_mm.
        3. Use Author_ID in all Tables.
        4. Need to document at the same time!
        5. Needed because of difficulties transferring special characters in file names.
      3. Enable "alternate names" for identically-named authors (to avoid middle-initial = X).
    • Backups
      1. Copying Errors:
        1. Investigate & fix directory error when backing up ZoomSearch database.
        2. This directory structure was set "not to back-up". Determine why this was ignored.
      2. Investigate Record-count discrepancies:-
        1. How do website files work as far as counts are concerned?
        2. Why aren't they recorded in Backup_History, nor the fact that the website was backed up?
        3. Different counts depending on whether new or old laptop is backed up. Investigate 63k discrepancy - lower on new laptop.
      3. Review architecture to improve performance; Need to document first
    • Books/Papers
      1. Investigate whether multiple Subject/Topic/Subtopic usage leads anywhere (ie. are just the first (of 3) actually used). Fix anything amiss.
    • Bridge
      1. As revealed by Spider: Bridge_Code_Archive.htm (14 items) & Bridge_Code_Archive_Recent.htm (5 items)
      2. Develop auto-reconciliation routines vs EBU results download
    • Documenter
      1. Investigate the error reports from the Documenter, especially unused variables & queries.
      2. Provide Functional Documentation for Website Generator (using Notes)
    • Education
      1. "Sitepoint (Learnable) - Sitepoint Learnable Web Development Courses": Plan what to do (and with the eBooks in my possession).
      2. Read "PC Pro - Computing in the Real World".
    • Infrastructure
      1. iCloud for Windows: Re-install & solve 'The upload folder for iCloud Photos is missing' problem. Try on new Laptop.
    • Notes
      1. Add "Note Alternates" to Note pages.
      2. Allow the option to concatenate Notes in the Printed version (ie. linearly embed them essay-style), rather than treating the hyperlinks as footnotes – but still keep the hyperlink & cross-referencing in place.
        1. For use as "disclaimers" - eg. for "Plug Notes".
        2. For Thesis / essays: the difficulty here is the need for linking passages to make the text run smoothly.
      3. As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
      4. As revealed by Spider: Blackberry links from archived Note 1010 (Maintenance Dashboard). 38 items.
      5. As revealed by Spider: Blog17 Achived Notes - sundry links to Printable Notes. Live Note OK. 100 items outstanding.
      6. As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
      7. As revealed by Spider: Sundry uncategorised. Refs failing. 30 items.
      8. Create Functor to facilitate addition of Audio Files to Music Page
      9. Investigate Note_Links: Section references seem to be incorrect
      10. Printable Notes: fix the bug whereby the “private” flag is round the wrong way.
      11. Suppress the publication of the Printable versions of Temp Notes
    • Photos
      1. Develop software & procedure to make adding more content to the photos pages easier to undertake.
      2. Timeline software: Add photos for Holidays
    • Process
      1. Determine why Recalculation & Changed Book/Papers produce unneeded regeneration.
      2. Full Website Regeneration is now taking 18.5 hours:-
        1. This is on my new laptop - it was taking 36 hours on my old laptop. Investigate why so, and improve performance!
        2. The end-to-end time on the new laptop is no better because the process pauses mid-way when the system goes to sleep overnight. I've set the power mode to 'Presentation', so we'll see if this fixes things.
        3. I got a copy error for private Note for Note 133. This contributed to extended run-times on the new laptop. Hopefully just a set-up problem.
    • Spider
      1. Analyse the results of the data collection exercise and design a plan of campaign to fix broken Internal links and prevent recurrence.
        1. Correct the code so the problems discovered by the Spider don’t recur.
        2. Delete 'orphan pages' that are never linked to, ie. Use the Spider to prune redundant pages18 automatically where possible.
        3. Fix the historical data where errors are uncovered by the Spider. An easier task now the site has a full-regen function.
      2. The Spider was generating WebRefs. Procedurally, this ought not to have been possible.
        1. The major problem turned out to be because unprocessed19 URLs got added to the end of the last WebLinks_Tester_Brief page, which then got Spidered. I've stopped this happening, so hopefully the problem will not recur. The fix was made in 18Q2.
        2. However, 4 other creations appeared - dated 18/05/18 - from the run of 10/07/18. The creation date was from the previous spider run, but the IDs show that they were produced in the latest run.
        I've re-opened the case!
    • Technology
      1. Look into Sistrix Smart20. Errors and warnings itemised are:-
        1. Duplicate content: seems to be variants on theotodman.com
        2. Title Tags: Empty, too long, identical
        3. Page Not Found
        4. Filesize in excess of 1Mb
        5. Meta-Description: Empty
        6. Few words on Page
        7. H1: Not used, used multiple times per page, identical across pages
        8. Pictures: Alt attribute missing
    • WebRefs
      1. As revealed by Spider: WebLinks_Tester_Brief_12.htm. Refs failing. 1 item.
      2. As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 117 items.
      3. As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 20 items.
      4. Documentation & Bug-fixes: Phase 2
        1. Re-document the procedures in the light of recent changes.
        2. Resolve issues generated / revealed by the spider.
        3. Investigate - and fix where possible - broken links.
      5. Reformat WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm
        1. Clarify 'truncated': Display, not link
        2. Allow more space for 'link returned', 'issue' and 'display text'
        3. The 'As Above" lines waste space. Only for Notes Archive? Consolidate onto single second line.
      6. Reformat WebLinks_Tester_Brief: Allow more space for 'link returned', 'issue' and 'display text'
  2. Other Websites: Priority 1 Items By Category:-

Summary of Progress to Date

This is hived off to various separate documents, which have now been harmonising and / or consolidated:-
  1. Summary of Progress to Date21.
  2. Outstanding Developments22,
  3. Functional Documentation23,
  4. A summary of time expended across the years developing my website24 is at "Software Development - Website - Development".



In-Page Footnotes:

Footnote 4: Footnote 5: Footnote 7: Footnote 9: Footnote 10: Footnote 11: Footnote 15: Footnote 18: Footnote 19: Footnote 20: Footnote 24:


Printable Version:



Table of the Previous 12 Versions of this Note: (of 75)

Date Length Title
10/10/2019 23:58:34 18052 Status: Web-Tools (2019 - September)
14/07/2019 20:29:46 16642 Status: Web-Tools (2019 - June)
05/04/2019 10:36:29 16128 Status: Web-Tools (2019 - March)
06/01/2019 23:36:58 18445 Status: Web-Tools (2018 - December)
10/10/2018 16:43:41 22079 Status: Web-Tools (2018 - September)
06/07/2018 18:56:10 16773 Status: Web-Tools (2018 - June)
05/04/2018 10:48:00 16588 Status: Web-Tools (2018 - March)
05/01/2018 00:11:31 13295 Status: Web-Tools (2017 - December)
09/10/2017 23:25:26 11848 Status: Web-Tools (2017 - September)
20/07/2017 14:34:05 11297 Status: Web-Tools (2017 - June)
10/04/2017 23:38:24 11470 Status: Web-Tools (2017 - March)
14/01/2017 20:18:14 13578 Status: Web-Tools (2016 - December)



Note last updated Reference for this Topic Parent Topic
19/01/2020 23:41:17 520 (Status: Web-Tools (2019 - December)) Status: Summary (2019 - December)

Summary of Note Links from this Page

Aeon Papers Internet Technology and Philosophy L. Messel & Co. Private Note Research - Internet Technology
Research - Proposal Theo Todman's Blog Website - Outstanding Developments (2020 - February) Website - Progress to Date (2020 - February) Website Generator Documentation - Control Page
Websites Maintained by Theo Todman        

To access information, click on one of the links in the table above.




Summary of Note Links to this Page

Internet Technology and Philosophy King's Maths Questions Simon - T1S1T1 Status: Consciousness Studies (2019 - December) Status: Priority Task List (2020: February)
Status: Summary (2019 - December), 2 Status: Summary Task List (2020: January - February) Status: Summary Task List (YTD: 19Q4 - 20Q1) Theo Todman's Philosophy Page Theo Todman's Website Maintainance History
Website - Progress to Date (2020 - February), 2, 3, 4, 5 Website Generator Documentation - Control Page      

To access information, click on one of the links in the table above.




Authors, Books & Papers Citing this Note

Author Title Medium Extra Links Read?
Software Development Website - Development Paper Low Quality Abstract    



Text Colour Conventions

  1. Black: Printable Text by me; © Theo Todman, 2020
  2. Blue: Text by me; © Theo Todman, 2020




© Theo Todman, June 2007 - Mar 2020.Please address any comments on this page to theo@theotodman.com.File output:
Website Maintenance Dashboard
Return to Top of this PageReturn to Theo Todman's Philosophy PageReturn to Theo Todman's Home Page