Theo Todman's Web Page - Notes Pages

Status Reports

Website - Progress to Date (2021 - December)

(Text as at 03/01/2022 23:58:34)


(For the live version and other versions of this Note, see the tables at the end)

Rationale for this Document

Items Completed In6:-

Items Completed by Category:
  1. Own Website: Completed Items By Category:-
    • Abstracts
      1. 20Q1
        • As revealed by Spider: Link to PaperCitings failing (2 items) - Timing issue
      2. 15Q4
        • Corrected data format on Identity Abstracts
      3. 07Q2
        • Added "Extra Data" parallel to Abstracts; ie. "More …" (Location etc.)
        • Created auto-backup of Abstracts, and more efficient maintenance web-routines
        • Segregated Abstracts into multiple directories
    • Aesthetics
      1. 15Q4
        • Reduced font + changed text in ubiquitous THEO'S PAGE heading
    • Animadversions
      1. 19Q4
        • Create Aeon_Files table & code to generate Aeon Page73 (to simplify processing and get round the 64k character limit).
      2. 07Q3
        • Added a "hot topics" link - things that I'm currently working on and want feedback ("Animadversions")
    • Architecture
      1. 21Q2
        • Created Check_Delimiters to protect against unpaired delimiters in lists, which squiff the formatting of documents that contain them. Output diagnostics to MsgBox, if not a re-gen, and to debug window to enable correction.
      2. 20Q3
      3. 20Q2
        • Enhanced the Functor processing to allow parameters. Previously, I had to create a new Functor ID each time for a very similar requirement. The main problem with this old method was that these Functors (where they call another Functor, or subroutine from a row in Functor) didn't feed through into the Documenter subroutines properly.
        • Truncated Cross_Reference_Changes table (from over 1M rows to under 50k) which should be self-maintaining, but isn't. This seems to have improved the regeneration times of Notes, Archived Notes and some other objects now that insertions into this table are quicker. Needs a permanent solution as part of my Cross-Referencing project.
      4. 18Q4
        • As revealed by Spider: Old links in unregenerated pages. 8 Items.
      5. 18Q3
        • Added 'TT' shortcut icon on all Webpage browser tabs
      6. 18Q2
        • Adjusted MS Access Control Page74 to feature new functions (Development Log & Automatic XRef) and consolidated the categorised Notes Lists into one button.
      7. 17Q4
        • Automated links between Quarterly reports (Task Lists)
      8. 15Q3
        • Made a lot of progress on improving the hyperlinking in my own website – I’ve made a good start on this project. The links now work accurately, but I need to ensure the efficient propagation of linkage changes across the site without requiring a complete regeneration.
      9. 15Q1
        • Vastly improved the “bulleting” options in Notes, etc.
      10. 14Q2
        • Added extra linkages between pages (eg. on Author).
      11. 14Q1
        • Repaired / updated my Website site-map
      12. 13Q3
        • Sorted conflicts between Subject, Topic & Sub-Topic IDs
      13. 13Q2
        • Improved error-recovery where encoding fails for links to Books, Papers and Authors. That is, where one of my bespoke tags (eg. +B75) doesn’t match with its corresponding anti-tag (eg. B+).
        • Replaced macros with code.
      14. 09Q2
        • Created Site-map for Website
      15. 07Q4
        • Added password-protection to selected pages
      16. 07Q2
        • Segregated database76 into Master and Slaves (code, papers, books, notes, web control, … vs PI)
        • Tidied up Footers throughout Philosophy Papers site (made into tables)
    • Authors
      1. 20Q2
        • Determined why the monthly regeneration process for Authors has ballooned from under 4 hours in January 2020 to over 17 hours in April 2020. Fix on 11/04/20 - time more than halved to 8.25 hours - by re-engineering the 'menu' process that counts the items linked to further down the page.
        • Enabled "alternate names" for identically-named authors (to avoid middle-initial = X):-
          1. Set up and populated Author_Name_Display
          2. Ensured Author_Name_Display appears on the Author page.
        • Further reduced the monthly regeneration process time for Authors from 8.25 hours to 13 minutes! Fixed by materialising and indexing a 'view' to table Authors_Cited_By_All_List.
      2. 20Q1
        • As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items. Dud page (Ontological Argument) deleted.
        • Restrict jump-table at top of Authors pages to relevant items, and include the item counts.
      3. 19Q4
        • Improve author-citations process:-
          1. Show author-citation counts on Author Summary pages.
          2. Only have an author-citation link on Authors pages if the author is cited!
          3. Import Authors from Aeon_Files table.
      4. 19Q3
        • As revealed by Spider: Author pages not regenerated when name corrected, leading to Sundry broken Links from other pages.
      5. 19Q2
        • Added Jump Table to sections of Authors Pages
      6. 18Q3
        • As revealed by Spider: 'Extra' Links to Notes from Citations List have the wrong directory (clumped in 1,000's like Papers rather than 100's). 7 items. Problem with AddCitations_List, so not just an issue for Authors.
        • Authors Narrative: enable footnotes.
      7. 15Q4
        • Corrected author's names (SN (FN)) for all papers and books
        • Provided easy links to works by the same author; Books, Papers; Joint authors
      8. 15Q1
        • Improved the format and content of “Authors” pages.
      9. 14Q2
        • Created the facility for Author Narratives.
      10. 11Q2
        • Created Author Cross-reference
    • Backups
      1. 15Q4
        • Enhanced my backup system to prune items backed up multiple times (eg. if they moved directory).
      2. 14Q2
        • Used a clone of my Spider routines to create a backup and restore system.
    • Books
      1. 20Q3
        • BookCitings references not sequenced correctly. Corrected.
        • BooksToNotes Link Pages: Re-engineered - used CreatePapersToNotesWebPages to generate pages
      2. 20Q2
        • Determined why the monthly regeneration process for Book Summaries had ballooned from 1.1 hours in January 2020 to 3.5 hours in April 2020. Partly fixed by by materialising and indexing the view (now table) Book_Citings_List_New, so now takes 39 minutes. Further improvement required, given that Paper Summaries only takes 5 minutes.
      3. 18Q3
        • As revealed by Spider: BookCitings. 405 items. Fixed problem with CreateBookCitingsWebPages not checking for change of Book during 'Repeats' processing.
    • Books/Papers
      1. 20Q4
      2. 20Q3
      3. 20Q2
        • Determined why the monthly regeneration process for Book-Paper Abstracts had doubled from 2.4 hours in January 2020 to 4.8 hours in April 2020. Successfully reduced to just upder one hour by materialising and indexing BookPaperAbstracts_List.
      4. 19Q3
        • Print Write-up Notes (not just the links) for appropriate Papers in BookPaperAbstracts
      5. 19Q1
        • Removed Blackberry processing and associated pages as no longer needed with fast broadband.
      6. 18Q3
        • Paper & Book Abstracts - Make 'Disclaimer' more prominent & show 'Paper Statistics' rather than 'Paper Summaries'
      7. 17Q4
        • New "Recent Acquisitions" Book List - Uncategorised
      8. 16Q1
        • Created direct links from Subject, Topic & Sub-topic (in Book and Paper Summaries) to their link pages.
      9. 15Q4
        • Neatened up the linkage headers in Book & Paper Abstracts.
        • Updated booklists to include shelf-identifiers
      10. 13Q3
        • Improved linkages between Book/Paper Lists and Summaries or Abstracts
        • Made enhancements to "Books & Papers Collection Categorised" Pages
      11. 13Q2
        • Developed "Books by Location" Pages
        • Improved the linkage between Papers and Books.
      12. 11Q4
        • Showed Percentage of Paper / Book Read
      13. 11Q2
        • Added “changed dates” to Paper and Book abstracts, and produced “recently changed” pages for them
        • Created Inter-Book-Paper Citings Cross-reference
      14. 09Q3
        • Completed a full 2-way link from Book/Paper Summaries to Notes.
        • Made a lot of changes to the way Books and Papers are categorised; in particular:-
          1. Added 2-level jump tables, to save the browser being crowded with hundreds of cross-categorisations in the same page.
          2. While it’s OK for Books, it seems to have a few potholes for Papers, as some of the links fail.
          3. The problem may be something to do with my economical way of suppressing redundant sub-categorisation - ie. where one or more of Subject, Topic and Sub-topic are the same.
      15. 09Q2
        • Added "quality marker" to Paper & Book Abstracts
        • De-duplicated Books / Papers to Notes links
        • Improved, and corrected errors in, the Blackberry-friendly formatted lists of Books and Papers.
      16. 09Q1
        • Created automatic links from Paper Summary to Book Summary
        • Enabled links from Books to the Papers contained therein
        • Enhanced Books to Papers links to include papers multiply held
        • Improved the Books / Papers to Notes links to include the right Version and Section (and historical Title) of the Note
        • Provided an automated link from Books / Papers to the Notes referencing them
      17. 07Q2
        • Added "read" column in papers / books lists
        • Added Book Comments and Abstracts (as "More…") to Book-lists
        • Added change of author or title in "Abstracts audit"
        • Added Flags for "laptop" and "photocopy"; removed document location from Comments
        • Added multiple Jump-levels for BB Lists (to keep pages small)
        • Created "BB-friendly77" lists: Papers (Full) and Books
        • Created "Identity Papers Full by Sub-Topic" list
        • Created "Identity Read Papers by Sub-Topic" list
        • Created "Papers by Title within Topic" list
        • Jump-Tables: added "by topic" option
    • Bridge
      1. 20Q1
        • As revealed by Spider: Bridge_Code_Archive.htm (14 items) & Bridge_Code_Archive_Recent.htm (5 items)
      2. 19Q3
        • Add Mountnessing archive links to Results
        • NGS Refinements
        • Synchronise Documentator with main website generator
      3. 18Q3
        • Reformated + Fixed Hand-dealing pages to CSS standards
      4. 18Q1
        • Corrected format of Links from Bridge Page
      5. 17Q2
        • Created hyperlinks from my consolidated results pages to the actual results
      6. 13Q2
        • Created “Consolidated Bridge Results” webpages (for my own results).
    • Clock
      1. 16Q2
        • Created a "chess clock" application to help time tasks. Not much use in the end.
    • Data
      1. 15Q4
        • Corrected data content on Identity Sub-topics
    • Documenter
      1. 20Q2
      2. 18Q4
        • Outstanding Developments by Category appearing all in Bold, and links to lists failing.
      3. 18Q3
        • Created Functor using Form_Documentation_Links to automatically add control-links to documentation Notes.
        • Fixed Bug where a variable has the same name as a procedure, and is thereby confused with one. Produced a warning query (Variable_Code_Name_Clashes) for manual correction.
      4. 18Q2
        • Investigated the error reports from the Documenter:
          1. Improved the diagostics
          2. Removed unused variables from the Generator code.
      5. 18Q1
        • Created consolidated Development Log, and updated "Progress to Date" and "Outstanding Developments" Reports
        • Documenter Bug-fixes: Objects in Comments, MsgBoxs & Debugs
      6. 17Q4
        • Code Archive: Created Web-pages of Recent Code Changes
        • Code Archive: Hadn't been working - fixed and recovered a selection from backups
      7. 16Q1
        • Created an automatic Forms documenter – with links to the code.
      8. 15Q4
        • Segregated the code-archive, which was bloating79 the generator database.
      9. 14Q2
        • Created multiple secure areas on my website, so that Documentation is no longer hiding under my Supervision Notes.
      10. 13Q2
        • Created a code & query archive facility as part of the Code Documenter.
        • Wrote the Code Documenter.
    • Footnotes
      1. 18Q3
      2. 10Q2
        • Added intra-Note footnotes – a real breakthrough (strange I’d not thought of this before). It will enable me to convert to Note form all the old .pdfs from my undergraduate days – which are replete with footnotes – if this is any use.
      3. 07Q3
        • Improved transparency of Footnote referencing in printed Notes
      4. 07Q2
        • Created auto-backup of Footnotes82, and more efficient maintenance web-routines
        • Created auto-renumbering83 routine for Footnotes
    • General
      1. 19Q2
        • Created 'Chess' Home Page
          1. Created as a main menu item and Note84
          2. Created consolidated results pages
          3. Added games & links to pgn4web
      2. 18Q2
        • Website Tidy-up:-
          1. Wrote a function to delete - with specified exceptions - items not regenerated in the last full website regeneration.
          2. Tidied up Site - deleted non-updated pages (especially old Printed Notes). Automatically on the local site, manually - using the audit log - on the live site.
          3. Investigated setting up an FTP link as a drive so I could automatically delete from the live site, but couldn't get it to work. Easy to set up as a drive, but not to assign a drive-letter so the drive can be used by VBA.
      3. 12Q3
        • Improved Titles for Search Results display optimisation
      4. 07Q2
        • Improved "Disclaimer85" Page. Reduce font, add ref to philosophy databases + hyperlinks
    • Infrastructure
      1. 19Q3
        • PDFelement: Purchase & Commission upgrade from Version 5 to Version 7 for $39 - await new Laptop
        • Purchase & Commission new Lenovo Laptop: awaiting replacement following motherboard failure!
      2. 18Q1
        • Migration of Website from UKServers to Namesco. Test out.
    • Notes
      1. 21Q3
        • Create a page to compare the vocabulary across related languages on Ling
        • Note Snippets
          1. Created a method of using the same text in multiple Notes.
          2. Created Note_Snippets table.
          3. Created Functor_24, with some control over the bullets.
          4. Currently only used in Thesis - Chapter 01 (Introduction)86 and General Surveys87.
          5. May require enhancement as it gets more widely used.
      2. 21Q1
        • Developed a mechanism to make the reading lists in my PID Notes easier to maintain and transfer to the Chapters on my Thesis:-
          1. Created a new table to hold the reading lists (include some static so it's easy to maintain): PID_Note_Reading_Lists
          2. Created a new Functor to use this table to generate the reading lists within a PID Note (automatically segregating "read" from "unread": allowed to run with two options to allow for comments between the lists: Functor_22, Options 6 and 7, respectively).
          3. Enabled footnotes for reading-list items.
          4. Created one-off queries (PID_Notes_Books_Referenced_Gen & PID_Notes_Papers_Referenced_Gen) to extract the reading lists from PID Notes and populate the new table (using the Cross_Reference table)
          5. To prove the process, started manually to replace the Reading Lists in all the PID Notes with the Functors (completing this will be a manual exercise and, after each run, will involve checking Live versus Test sites).
          6. Procedurally, in the Thesis Chapters, if Notes are referenced in the Text, use the "NP" encoding so that they don't contaminate the reading lists: only those explicitly listed should do so.
          7. Added flags to either exclude Books or Papers logged in PID_Note_Reading_Lists altogether, or to segregate them as "missed the cut" in the Thesis Chapters (and, possibly) the Notes themselves.
          8. Automatically update table PID_Note_Reading_Lists to maintain the "read status" and Book/Paper Title.
        • Developed a mechanism to monitor usage of PID Notes within my Thesis (by Chapter):-
          1. Created Note 131788, which incorporates as table and a list, created as follows ...
          2. Used Notes: Functor_21, option 14 (Query PID_Notes_Used_By_Thesis_Chapter_Xtab)
          3. Unused Notes: Functor_22, option 5 (Query PID_Notes_Unused_By_Thesis)
        • Enabled multiple parameters for Functors
        • Enhanced Functors to work for selected non-temp Notes (ie. PID Notes: ensured Functor_22, Options 6 and 7 can run on a non-Temp Note, without creating an Archived Note; borrowed code from Auto_Reference_Notes) so that up-to-date stats can be incorporated. This was fairly complex as I wanted to avoid production of an Archived Note each time a non-Temp Note is regenerated. I also wanted to ensure that Notes whose variable text consists entirely of Functors get archived correctly (which they might not have if I'd removed Functor-generated text from Notes before saving them to the Notes table; in fact, the text saved to Live and Archive is identical and includes the full Functor content).
        • Enhanced the mechanism to make the reading lists in my PID Notes easier to maintain:-
          1. Made the table PID_Note_Reading_Lists easier to maintain: added a menu at the start of the Create Notes Web Pages89 to (inter alia) open the PID_Note_Reading_Lists table, add or delete rows, change the "excluded" and "cut" flags, and amend Footnotes. Book/Paper titles and read-status are automatically maintained by the Functors.
          2. Added functionality to allow the reading lists to be further segregated - as in the Notes Animalists90 and Christian materialism91 - which din't use the Functors (though their reading lists are on PID_Note_Reading_Lists).
          3. Created a further Functor (Functor_23, Option 17) to generate the Thesis Chapter Reading Lists from the Notes explicitly listed as relevant in the new table Thesis_Note_XRef. Uses Functor_Indented_List, which also creates (via Functor_23, Option 16) the indented PID-Note lists. A major triumph!
        • Modified Functor_21 Option 3 to allow multiple Audio Files per timesheet line
      3. 20Q4
      4. 20Q3
        • Added Aeon Abstract (ie. Footnote) link to Aeon Webref items in Summary task List reports for 'read' items
        • Added option in Auto_Reference_Notes to - for Live (ie. not TEMP) Notes - update the latest Archived Note to equal the updated Live Note, hence avoiding superfluous archives.
        • Added option in Auto_Reference_Notes to only confirm new items (leaving previously-flagged items untouched)
        • Corrected Issues with Nested Functors: Added Functor_ID to End-tag
        • Determined why the run of Auto_Reference_Notes_Regen took 2.5 hours on 02/09/2020 (and climbing). It was down to the Aeon Note95 being searched; in particular Check_If_In_Container, checking for items in Footnotes and / or Quoted Text. As this Note is regenerated, there's no point auto-linking it. Time reduced to 23 minutes once this Note - and other Notes of inappropriate Note Groups - was removed.
        • Highlighted Archived Notes clearly as not the latest
        • Upgraded Auto_Reference_Notes to allow a parameterised 'termination or continue' for very long update runs
        • Upgraded Auto_Reference_Notes to avoid key-words within +AA+-style Authors names (using Check_If_In_Container).
        • Upgraded Auto_Reference_Notes to log its actions to a new table (Auto_Reference_Notes_Actions) so that any errors can be investigated and improvements made.
      5. 20Q2
        • As revealed by Spider: Sundry uncategorised. Refs failing. 30 items. Fixed manually.
      6. 20Q1
        • As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
        • As revealed by Spider: Blackberry links from archived Note 1010 (Maintenance Dashboard). 38 items.
        • As revealed by Spider: Blog96 Achived Notes - sundry links to Printable Notes. Live Note OK. 97 items.
        • As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
        • Solve the problem of the size limitation for Notes by allowing their concatenation.
      7. 19Q4
        • Create procedures (manual and via Functor_17) to add Audio Files to Notes
      8. 18Q4
        • As revealed by Spider: Consciousness Status Report97 Achived Notes - links to BookCatalogCategorised_16_115_471.htm failing. Live Note OK. 46 items. 24 items outstanding.
        • As revealed by Spider: Links from Blog in Level 1 or 2 Printed Notes 51298 and 98199 failing. 141 items.
        • As revealed by Spider: Links to sundry 'static' pages failing.
          1. 13 items fixed.
          2. 7 items outstanding: to do with Pete's PhD Test site. Links seem to be OK, so may be a problem with the Spider. Directory "Petes_PhD" was the only one set to "do not parse". The file wasn't in the site map. Reset; hopefully it'll work!
        • As revealed by Spider: Status Reports - Achived Notes - sundry links failing. 50 items in total. Live Notes fixed.
          1. Website: Maintenance Dashboard100. 14 items.
          2. Personal Identity Status Report101. 11 items.
          3. Summary Status Report102. 4 items.
          4. Actual Detail Summary Report103. 6 items.
          5. Chess Status Report104. 12 items.
          6. Etc. 3 items.
        • As revealed by Spider: Sundry links failed for Supervisions. 30 items.
        • Create Priority Task List Report105.
      9. 18Q3
        • As revealed by Spider: Bridge Status Report106 Achived Notes - sundry links failing. Live Note OK. 70 items.
        • As revealed by Spider: Languages Status Report107 Achived Notes - links to Dissertation failing. Live Note OK. 43 items.
        • As revealed by Spider: Religion Status Report108 Achived Notes - sundry links failing. Live Note OK. 59 items.
        • As revealed by Spider: Links to Notes_xxx_Links, from Abstracts (94 items), Notes (25 … corrected manually), NotesPrint (1916). Fixed by +LL+ functor implementation.
        • As revealed by Spider: Notes_List_Control links fail for Supervisions. 58 items.
        • Automatic Key-word Hyperlinking to Notes Project: Phase 2
          1. Added links based on Book & Paper Titles
          2. Added ***Read***-marker to Lists
          3. Allowed links to be ignored (rather than suppressed) during approval process
          4. Skip links to Status Reports, etc.
        • Blog: Created Functor to generate Blog from Blog.
        • Created auto-regen of all Note Link (Note_nnn_Link) Pages
        • Created new '+LL+' Functor for links to Note_Link pages; fixed associated Notes.
      10. 18Q2
        • Added a process to 'fix' - by adding the Note-ID - those note-links added without stating the ID. This will save processing time.
        • Created automatic key-word referencing (ie. to Notes in the Notes-Group "Personal Identity"). This was complicated for a number of reasons:-
          1. There are always false positives, so there needed to be a way of "proofing" what's to be done before doing anything.
          2. There will be times where it is inappropriate (eg. in quoted work by other authors). So, there had to be a way of permanently suppressing such links.
          3. The referencing differs depending on whether the term is in a Footnote or not.
          4. I needed to check for already-existing links, variant grammatical forms, etc.
          5. I created a MsgBox routine - providing context - so that what needed to be done with potential links could easily be decided when the linking process was (re-)run.
          6. Links are always to Notes, but are from Notes, Archived Notes, Authors and Book & Paper Abstracts & Comments.
      11. 18Q1
      12. 17Q4
        • Added %age complete to Notes Reading Lists
        • Enabled deduction of Note link by omitting the Note ID from the mark-up and doing a look up on the key-word and its varients.
      13. 17Q3
        • Added Colour-coded "quality markers" to Notes: included them in lists and jump-tables
      14. 16Q1
        • Allowed Note- & Footnote-links from multiple words116. This is such a useful feature!
        • Completed converting “static” pages to Notes.
        • Enabled direct links to Notes from within Footnotes.
        • Stopped (I hope!) “private” Notes showing in Paper Abstracts.
      15. 15Q4
        • Converted the “static” pages in my website to Note format (so they can change and retain history).
        • Fixed problem of images not appearing in "Printed" Notes (wrong directory level) unless the images are repeated
      16. 15Q3
        • Tidied up the archived Notes.
      17. 14Q4
        • Made improvements to the regeneration and referencing of archived Notes.
      18. 14Q3
        • Created a progress-tracker117 for my Thesis.
        • Improved error-recovery where encoding fails for Notes links.
        • Started to split Paper Abstracts from Write-ups.
      19. 14Q2
        • Wrote routines to import old (but missing) archived Notes into the database so they can be (and have been) re-exported in the current format.
      20. 14Q1
        • Made a lot of improvements to my Site generators, in particular the pesky bug that stopped the links to archived Notes working correctly.
      21. 13Q3
        • "Single Image" development for Notes
        • Allowed "Authors Works Lists" to be linked-to from Notes
        • Created Archived Notes Re-Export facility
        • Simplified / Re-engineered Notes processing
      22. 13Q2
        • Improved "Concatenated Notes" production
      23. 10Q3
        • Add “Date Created” to Note_Links (for audit purposes; I’m still not convinced that links disappear when they should).
        • Investigate Note_Links: needs versioning, and section of calling Note
        • Wrote routines to automatically generate the “Printable Notes” footer – and to create the corresponding web pages – from the control table. Very useful.
      24. 09Q4
        • Restricted the size of the “previous versions” tables for Notes to 12.
      25. 09Q3
        • Enabled automatic immediate promotion to Live Website of selected Temp Notes.
        • Enabled Concatenated Notes (by Group). This is only allowed for “Identity” Notes at the moment, which is what I’d wanted it for. It allows all Notes for a Group to be printed off in Title sequence, for ease of review.
        • Ensured that the Notes Reading-list is generated from all 3 sub-topics.
        • Functionally enhanced the Notes-generation processing options; to make niftier production possible. In particular, where individual Notes are selected, I made the re-generation of the cross-references optional (so that if only textual variation is being iterated, the iterations are quicker).
        • Improved the performance of “All Notes” regeneration. The problem was with the generation of the generic reading lists based on Note Title. These only apply to the “Identity” Notes, and it was taking a long time to determine there was no list to produce.
      26. 09Q2
        • Added a table of Notes history at the end of each Note (ie. not just previous version) - to save tedious chaining
      27. 09Q1
        • Added reading-list (of works cited) at the end of the Note, as well as in the Printed Note
        • Created optional "don't print" feature for specific footnotes.
        • Fixed the problem of the overpopulation of the Notes directory
        • Option for Notes-Regen:
          1. Changed-only
          2. Changed, and all impacted by Change
          3. All
      28. 08Q1
        • Allowed regeneration of Temp Notes only
      29. 07Q4
        • Added referenced Books / Papers as footnote jump table to Notes
        • Added referenced reading lists to printable version of Notes
        • Improve "Temp" note processing. Suppress "_T" suffix (just don't copy to Transfer)
        • When a Note is archived, ensured a printable note is also archived118 (if this Note has a print)
      30. 07Q3
        • Created "respondent notes" - database identifier & red indicator + text
        • Created119 a "quick lookup" table of responses to a note and its sub-notes (recursive function on Notes_Links)
        • Enabled generation of printable version of Notes
        • Made Note-change-history available on-line (added status + publishable flags)
      31. 07Q2
        • Added "footnote" pages120 for Carthusians & King's
        • Added "outgoing links" summary footer to Notes pages
        • Allowed two sorts of Note - document-specific ("Write-up") and general
    • Papers
      1. 21Q2
      2. 20Q3
      3. 20Q2
        • Determined why the monthly regeneration process for Paper Abstracts had ballooned from just over 6 hours in January 2020 to over 17 hours in April 2020. Reduced to just under 4 hours as a result of materialising and indexing the view (now table) Paper_Citings_List_New. Further improvements required.
        • Determined why the monthly regeneration process for Paper Summaries had ballooned from just over 3.5 hours in January 2020 to around 11.5 hours in April 2020. Fixed by materialising and indexing the view (now table) Paper_Citings_List_New, so now takes 6 minutes!
      4. 20Q1
        • Make use of Comments for large Abstracts seamless by appending Comments (if > 5k) to Abstracts.
      5. 18Q2
        • Papers associated with broken links:-
          1. Created an area to hold pdfs, and automated links thereto.
          2. Created a pop-up to say I'd remove such pdfs if requested by their authors.
          3. Consequently, haven't password protected the area yet.
      6. 18Q1
        • Correction of narratives on Note time-recording pseudo-papers
    • Photos
      1. 19Q1
        • Created Photo-logging usage via Images table.
        • Created Timeline software
        • Timeline software: Added photos of Coxes Farm
      2. 18Q3
      3. 13Q2
        • Reduced the size of photos (to make the pages quicker to load).
    • Process
      1. 20Q3
        • Determined why Recalculation (cmdRecalculate_Click) took so long: it varied depending on how many Notes were regenerated, but seemed to be 17.5 minutes even if there are none. I think it was down to a problem with nested Functors leading to Notes growing inordinately: in particular the development logs. Now takes around 5 minutes.
      2. 20Q2
        • Full Website Regeneration took 60 hours in April 2020. This has now been substantially fixed, as of end June 2020, in that regeneration now takes just under 11 hours. While further improvements are possible, the best use of time is to avoid the need for regular regeneration altogether by completing my Cross-referencing project.
        • Corrective work was split out into the items requiring improvement, must of which have now been implemented.
        • History and analysis:-
          1. This 'balooning' was on my new laptop which has solid state disks - it was taking 36 hours on my old laptop, then halved when I got the replacement, but then nearly doubled!
          2. I investigated why this was so, and improved performance mainly by putting in a trace (using a timer - GetTickCount - found on the web that allows logging of elapsed time in milliseconds, using StartTimer and EndTimer) and finding the 'pinch points'
          3. I suspected that it was a new release of MS Access that's caused a problem; maybe a lost index or something similar. I couldn't find any evidence of this, but a mumber of queries were taking a second or so to execute, which small times multiplied up drastically when the number of pages on my site - over 100k - are taken into account. Quite why a cliff-edge had been reached, I know not!
          4. Three areas had been particularly impacted: Authors, Paper Abstracts & Paper Summaries, which have all trippled or worse. These are covered as seperate developments under 'Authors' and 'Papers'.
          5. Book Summaries and Book-Paper Abstracts had also at least doubled, but this has had less of an impact as they took much less time in the first place.
          6. Other regeneration processes - in particular 'Notes' - didn't appear to have been impacted.
        • Wrote Check_Database_Size to check that the size of the database isn't at risk of breaching the 2GB limit. Required because of the need to re-materialise various views that have been instituted to speed up the full website regeneration routines, and which need to be run each time cmdRecalculate_Click is run, which is often.
      3. 19Q4
          1. Full Website Regeneration failed in Notes Regen because the first Private Note (133122) couldn't be copied from the Transfer directory as the private note hadn't been created there by then.
          2. It should have been, from looking at the code.
          3. Also, the private note itself (1163123) is not regenerated until later.
          4. In the start-December 2019 run, no such problem occurred. Maybe the problem was caused by copied between laptops. So, I've flagged it as complete for now, pending a recurrence.
        • Fix Functors to enable previous quarter's Status Reports after Time period roll-over
      4. 18Q4
      5. 18Q2
        • Re-engineered Website Maintenance Dashboard125 to show extra topical links (and show them at the top of the page).
      6. 18Q1
        • Improved efficiency of daily changes regeneration (removed msgboxes)
        • Wrote a routine to update a set of .htm files, replacing text in them all
      7. 15Q4
        • Created a consolidated processor to enable overnight regeneration of the whole website.
      8. 14Q1
        • Created a “Website Regeneration Dashboard” to show when the various elements of my website have been refreshed, and how long the routines took to run. Links to this appear in the footer to each webpage.
        • Improved the regeneration routines for the various classes of webpage, and got them to record statistics in support of the “Website Regeneration Dashboard”.
      9. 09Q4
        • Set up my database system to roll forward to the next academic year, and generate web-pages appropriately.
      10. 09Q3
        • Created a “performance stats” database and routines to write to it – though I only implemented them for the Notes generator.
      11. 07Q2
        • Improved ease of FTP process by use of Transfer Directory
    • Search
      1. 17Q2
        • Wrote routines to create a cut-down Website excluding pages with irrelevant or duplicated information. Used this to create the ZoomSearch facility.
      2. 07Q4
        • Created search facility using ZoomSearch
    • Spider
      1. 21Q3
        • Improved Spider performance by removing all Printable Notes - this will need to be reversed sometime.
      2. 20Q4
        • System Resources Exceeded - "Run Time Error 3035" - re-emerged in Full_Link_Same_Directory_Gen. Last month I had replaced query Full_Link_Same_Directory_Updt with identical SQL, which had worked OK. This time I removed the "WHERE" statements and put them in the code itself. Seems fine.
      3. 20Q3
        • System Resources Exceeded - "Run Time Error 3035"; query Full_Link_Same_Directory_Updt in Full_Link_Same_Directory_Gen. Fixed by using SQL rather than a query.
        • The size of the main database was bloating to over 1.5Gb during the spider run, so was approaching the 2Gb limit.
          1. Used Check_Database_Size, with a parameter, to monitor the size of the main database and added a message similar to those reporting the compact / repair of the Slave database each time a status message is printed.
          2. Based on the diagnostics produced, it was found that the bloating occured at two stages:-
            → In Spider_Scurry: during the creation of Raw_Links_Temp_Temp.
            → In Spider_Copy: during the creation of Raw_Links_Temp.
          3. Fixes:-
            → Put check in Check_Database_Size to STOP if the database is over 1.5 Gb (parameterised via Max_Database_Size).
            → Moved Raw_Links_Temp_Temp to the Slave database.
      4. 18Q4
        • Determined why Full_Link_Up_Levels_Gen takes so long - approximately 4 hours. Re-architected to reduce database reads.
      5. 16Q4
        • Added “.shtml” and “.shtm” to the list of web-suffices.
      6. 15Q4
        • Finally cracked the Spider for external links, and used it to check the 3,000-odd links from my website. Some tidying up outstanding.
      7. 14Q4
        • Fixed the bug wherein the Spider ran out of system resources on large runs: now compacts & repairs the slave database as it does along (every 200k links; parameterised).
      8. 14Q3
        • Continued working on my Spider and started to use it to clean up my website by diagnosing broken links.
      9. 13Q2
        • De-bugged the Spider.
      10. 10Q2
        • Wrote my own Web Spider.
          1. It’s a little bit of a cheat, in that it doesn’t actually invoke the web-links, but crawls through my directory structure selecting all the .htm files, and rummages through the HTML looking for links which it then copies to a table.
          2. It also maintains a table of the directory structure and files therein, and then works out the real address from the relative addressing.
          3. It then allows cross-checking between the links and the files.
          4. Links external to the site have to be investigated manually (at least I have a list of all the links) – unless I can work out how to use the list to power a browser to check.
          5. There are about 333k links in 55k files – the full run took about 8 hours.
          6. It now runs incrementally based on file last-updated dates.
    • Status
      1. 21Q3
      2. 20Q4
      3. 20Q3
        • Added Weekly Project Plans to Priority Task List
        • Created web-page (using Functor_21) showing oboe practice hours by work played
        • Quarterly Project Reports: Corrected Functor_08. The Project Planned YTD % kept having to be bodged!
      4. 19Q2
        • Corrected Quarterly Reporting for uneven Plan (Holidays)
      5. 18Q4
      6. 18Q3
      7. 18Q2
        • Added extra items to Functors to facilitate Quarterly Reporting:-
          1. Automatically calculated QTD & YTD time statistics in the quarterly reports.
          2. Provided a YTD task list for Status Reports for projects "in abeyance" to make reporting easier.
          3. Modified the statistics - to be just YTD - for YTD Status reports.
          4. Created table for - by Project - "Plans for the near future" tasks, and a functor to add them to the status report.
          5. Took "items competed" and "plans for the near future from the development log for the Web-tools status report.
          6. Created another functor to add these lists to the Summary report.
      8. 18Q1
        • Created web-page generator for Development items (ie. Created the Web-Project "To Do" List and "Outstanding Developments" directly from this list).
      9. 13Q3
        • Re-engineered Monthly Reporting on Change of Year
      10. 13Q2
        • Improved "This Month's Summary" Reporting
        • Improved Quarterly Reporting
      11. 13Q1
        • Re-engineered Monthly Reporting Tables & improved Monthly Reporting
    • Technology
      1. 20Q4
        • Consolidated Site-Hits:
          1. Reviewed Webalizer Methodology & Consolidated Stats
          2. Aggregated Robot URLs
      2. 19Q1
        • Investigated the Hit Stats provided by Webalizer and produced web-pages to show the most active pages.
      3. 13Q3
        • Converted Tables to CSS3 standard
        • Used CSS in all manually-crafted and automatically-generated pages
      4. 13Q1
        • Standardised to HTML5134
    • Timesheets
      1. 17Q4
        • Separated Time Recording for Aeon
      2. 15Q1
        • Segregated time-recording so I can easily see how much time is spent developing and supporting my own website, and how much is spent on those of other organisations.
      3. 13Q3
        • Re-engineered Time-Recording, so that actuals are held against Papers (and Books) in a separate table, rather than as columns by year in the Papers / Books tables themselves. Retained the grand totals only.
      4. 08Q4
        • Updated Book totals from totals of the Papers contained therein
    • WebRefs
      1. 21Q1
        • Webrefs_Update failed because IE looped with Aeon / Psyche pages. Performed manual checking for a while, but then Aeon seem to have fixed their pages so the problem seems to have "gone away". It doesn't occur for other pages.
      2. 20Q3
        • Improved WebRefs checker (Webrefs_Update) further to check for Error 403 "Forbidden". This involved finding way of checking pdfs where the returned page is in fact HTML or XML (see DevLog Ref 379).
        • Reformated WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: The 'As Above" lines waste space. Consolidate onto single second line.
        • Reformated WebLinks_Tester_Brief, WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: Allowed more space for 'link returned', 'issue' and 'display text' and added Explanation column
      3. 20Q2
        • Webrefs_Update: Improved the processing of this sub which controls IE to check the URLs corresponding to WeRefs are still valid.
          1. Improved the recovery processing after IE fails or becomes detached. Now seems to work perfectly!
          2. Improved processing for 404 (page not found) errors - detect them where URL returned differs from that requested (so the error is correctly categorised) and also where it is the same (so the error is detected) - by using the GetElementsByTagName method to look through Title and H1 elemnts (where they exist).
          3. Note that there is still an issue for pdfs where the above methods don't work; though the page returned is an HTML or XML page, the changed URL still claims to be a pdf, so I can't check in case it's a real pdf and the process fails.
        • Amend WebRef 'Name' link to #Off-Page_Link_WxxxW style. Also amend Quarterly and Annual task lists to use this reference.
        • Amended WebRef 'Name' links to #Off-Page_Link_WxxxW style so referencing from other pages is possible (useful for Aeon)
        • As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 184 items. res://ieframe.dll/ in Returned_URL.
        • WebLinks_Tester: If the URLs of WebRefs are changed between Spider runs, the URL mapping fails, and the WebRefs appear at the end of the last page, wrongly categorised. Fixed to ensure they appear on their own page, added to the jump table as 'WebRef Missing' and added an explanation.
        • WebLinks_Tester_Full & WebLinks_Tester_Full_Map: Reformated Jump Table to 30 columns and multiple rows.
      4. 20Q1
        • As revealed by Spider: WebLinks_Tester_4.htm. Refs failing. 32 items. res://ieframe.dll/ in Returned_URL.
        • As revealed by Spider: WebLinks_Tester_Brief_12.htm (etc). Refs failing. 41 items. res://ieframe.dll/ in Returned_URL.
        • As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 51 items outstanding. res://ieframe.dll/ in Returned_URL.
        • WebRefs checker failing to detect broken links for Aeon - if a link fails, I get the message 'The page you were looking for could not be found (404)', but since the URL doesn't change, I don't spot it. This may happen for other sites as well. Fixed for Aeon - watch out for others.
      5. 19Q3
        • As revealed by Spider: Hits_Pages_Total_Crosstab_Descending.htm.
      6. 18Q4
        • Added Display text to WebRefs for Links to Stanford, Wikipedia & Aeon
      7. 18Q2
      8. 18Q1
        • Determined & reported usage directly using embedded +WnnnW+ encoding rather than via the Spider. Reconciled the two methods.
        • Documentation & Bug-fixes: Phase 1
          1. Completed "as is" functional documentation so I could see what's going on.
          2. Re-engineered so it made sense and ran efficiently without bugs
          3. Fixed the obvious data errors that confused the statistics - ie. where the returned URL differs from that requested and https issues.
          4. Wrote / improved diagnostic reports (web pages)
      9. 16Q1
        • Some tidying up of the 3,000-odd external links from my website.
      10. 13Q2
        • Some pages render oddly in Chrome (and maybe other browsers) – investigated136.
      11. 10Q3
        • Improved the functionality for automatically generating external Web-links in Notes, Book Abstracts and Paper Abstracts. Corrected current data to accommodate the new code.
      12. 07Q2
        • Made Web Refs in Comments / Abstracts into Hyperlinks
  2. Other Websites: Completed Items By Category:-
    • Acts
      1. 16Q3
        • Developed a website for displaying the textual and grammatical analyses and appendices of Pete’s PhD on the Acts of the Apostles. Two versions: Live (Acts: Live Site (Defunct)) and Test (Acts: Test Site).
    • Bridge
      1. 21Q3
        • Major Spring-clean of the Mountnessing Bridgewebs site.
      2. 17Q1
        • Added new features (70s Club & Monthly Pairs Winners) to my Bridge mini-websites.
        • Created Master Point Lists & Promotions for the ECBA website.
      3. 16Q3
      4. 16Q1
        • Accommodated Alaric’s new scoring system for St. Edith’s.
        • Created Ladies’ and Gentlemen’s ladders for my mini-websites.
        • Reconciled my X-IMP calculations with a new feature provided by Bridgewebs.
      5. 15Q4
        • Segregated the Bridge website generators from those for my own website. They now live in separate databases and have their own documenters.
      6. 15Q3
        • Created a new website (using Bridgewebs) for the First Class Bridge Academy (First Class Bridge).
        • Ported my “handicap / improvers” routines from Hutton to “Mini Websites”.
      7. 15Q2
        • Development of the “Eve Goblets” (victor ludorum) software for the ECBA, and roll-over to the new season.
      8. 15Q1
        • Further improvements to my Butler-IMPs and Cross-IMPs calculation routines.
        • Started a project to provide small websites for bridge clubs without sites: Essex Bridge Results.
        • Took on the ECBA Website (ECBA Website (Defunct)), at the cost of considerable effort by way of reformatting & tidy-up.
      9. 14Q4
        • Improvements to my Butler-IMPs and Cross-IMPs calculation routines – different formats, including the Jeff Smith format, and extension to Butler-scored Pairs events. This work has been taken up by the ECBA selectors for county-team selection.
      10. 14Q3
        • Created a suite of programs to extract data and calculate Butler-IMPs and Cross-IMPs from the travellers on Bridgewebs Teams events, and output ranking-lists to the Web.
      11. 13Q2
        • Merged137 Philosophy & Bridge Databases
    • Enigma
      1. 18Q4
        • Added 'EE' shortcut icon on all Webpage browser tabs
      2. 15Q3
        • Created a small website for a music group Julie and I attend – the Enigma Ensemble (Defunct).
    • Sophie
      1. 19Q3
      2. 18Q4
        • Added 'SB' shortcut icon on all Webpage browser tabs
      3. 15Q3

In-Page Footnotes

Footnote 6: Footnotes 40, 80: Footnotes 41, 81: Footnotes 50, 109: Footnotes 57, 116: Footnotes 58, 79: Footnotes 59, 138: Footnotes 61, 75: Footnotes 62, 136: Footnotes 63, 137: Footnotes 64, 134: Footnotes 65, 118: Footnotes 66, 119: Footnotes 67, 76: Footnotes 68, 77: Footnotes 69, 70, 72, 82, 83, 120: Footnotes 71, 85:

Live Version of this Archived Note

Date Length Title
01/07/2024 06:47:31 127200 Website - Progress to Date (2024 - July)

Table of 12 Earlier Versions of this Note (of 21)

Date Length Title
02/07/2021 20:32:38 111171 Website - Progress to Date (2021 - June)
03/04/2021 19:58:05 109399 Website - Progress to Date (2021 - March)
04/01/2021 19:40:48 101369 Website - Progress to Date (2020 - December)
02/01/2021 14:29:46 100708 Website - Progress to Date (2020 - December)
04/10/2020 00:27:22 95240 Website - Progress to Date (2020 - September)
03/07/2020 22:09:07 86201 Website - Progress to Date (2020 - June)
04/04/2020 00:14:24 72717 Website - Progress to Date (2020 - March)
19/01/2020 23:41:17 69093 Website - Progress to Date (2019 - December)
10/10/2019 23:58:34 66813 Website - Progress to Date (2019 - October)
14/07/2019 20:29:46 65148 Website - Progress to Date (2019 - June)
05/04/2019 10:36:29 70085 Website - Progress to Date (2019 - March)
06/01/2019 23:36:58 68922 Website - Progress to Date (2018 - December)

Table of the 6 Later Versions of this Note

Date Length Title
04/10/2023 21:59:02 124304 Website - Progress to Date (2023 - September)
06/07/2023 00:43:12 123503 Website - Progress to Date (2023 - June)
30/04/2023 01:06:14 122764 Website - Progress to Date (2023 - March)
10/01/2023 23:35:31 119468 Website - Progress to Date (2022 - December)
10/10/2022 22:32:10 116821 Website - Progress to Date (2022 - September)
03/07/2022 01:29:17 116479 Website - Progress to Date (2022 - June)

This version updated Reference for this Topic Parent Topic
03/01/2022 23:58:34 822 (Website - Progress to Date (2021 - December)) Status: Web-Tools (2024 - March)

Summary of Notes Links from this Page

Aeon Papers (03/01/22) (4) Animalists (2) Brief Thoughts on Language & Languages (2) Carthusians - Main Text (2) Christian Materialism (2)
General Surveys (2) L. Messel & Co. (2) PID Note, Book & Paper Usage (2) Private Note (2) Status: Actual Detail Summary (2007 - 2022) (2)
Status: Bridge (2021 - December) (2) Status: Chess (2021 - December) (2) Status: Consciousness Studies (2021 - December) (2) Status: Languages (2021 - December) (2) Status: Personal Identity (2021 - December) (2)
Status: Philosophy of Religion (2021 - December) (4) Status: Priority Task List (2021 - December) (6) Status: Summary - Actual versus Plan (2021 - December) (2) Status: Summary (2021 - December) (4) Status: Summary Task List (2021: October - December) (2)
Status: Summary Task List (YTD: 21Q4) (2) Status: Thesis Dashboard (2021: September) (2) Status: Web-Tools (2021 - December) (5) Theo Todman's Blog (2) Theo Todman's Chess Page (2)
Thesis - Chapter 01 (Introduction) (2) Website - Outstanding Developments (2021 - December) (9) Website - Progress to Date (2021 - December) (8) Website Generator Documentation - Control Page (3) Website Generator Documentation - Create Notes Web Pages (2)
Website Generator Documentation - Cross-Referencing (2) Website Generator Documentation - Functors (2) Website Generator Documentation - Web Links (2) Website: Maintenance Dashboard (03 January 2022) (6)  

To access information, click on one of the links in the table above (if any).

Summary of Note Links to this Page

Status: Web-Tools (2021 - December) Status: Web-Tools (2022 - March) Website - Outstanding Developments (2021 - December) Website - Outstanding Developments (2022 - March) Website - Progress to Date (2021 - December), 2, 3, 4, 5, 6, 7, 8

To access information, click on one of the links in the table above (if any).

Text Colour Conventions

  1. Blue: Text by me; © Theo Todman, 2024

© Theo Todman, June 2007 - July 2024.Please address any comments on this page to output:
Website Maintenance Dashboard
Return to Top of this PageReturn to Theo Todman's Philosophy PageReturn to Theo Todman's Home Page