Theo Todman's Web Page - Notes Pages
Status Reports
Website - Progress to Date (2025 - February)
(Work In Progress: output at 16/02/2025 23:42:43)
Rationale for this Document
Items Completed In6:-
- 25Q1
- Own Website
- Bridge:
- Bridge Results Export: Correct residual problems following code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug. To fix, I simply added a check that if the link to the sessions results for the final row isn't blank, I issue a warning.
- Other Websites
- A1Wasps:
- a1waspsandbees: Invent method of getting the site higher up the Google search list without paying!
- Bridge:
- Small Systems: Make code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug. I improved the fudge that simply re-creates Small Systems for the current year by removing the hard coded '2024' and replaced it with 'Current Year'. Should be adequate. Seems to work fine.
- Old House Bridge Club Website: Set up for 2025 and permanently resolve the 'too many databases' problem.
- Enigma:
- 24Q4
- Own Website
- Backups:
- Purchase & commission new 5Tb WD Backup Drive. Copy 2Tb backup drive contents across.
- Books/Papers:
- Bridge:
- Correct processing for Partner's NGS calculation
- Infrastructure:
- Migrate 2 x Laptop + Desktop from Norton AV to Microsoft Defender: Norton non-renews 13/11/24
- Notes:
- Active Note Groups with Notes Updated in the last 7, 14, 28 days: producing incorrect results. Important as there are links from my Home Page. Added Temp_Note_Timestamp column to Notes table.
- Other Websites
- A1Wasps:
- 24Q3
- Bridge:
- Bridge Results Export: Make code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug. The symptoms:-
- After run, while the database closes OK, MS Access remains active and had to be crashed via Task Manager.
- If the Export is run a second time, as is required for Partner's calculated NGS to show correctly, without closing the database first, we got 'permision denied' when trying to create Bridge_Results_Theo.htm.
To fix, I only create the hyperlink to the results if the link is blank. This is a bit of a bodge, and I may revisit it later.
- Small Systems: Make code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug.
- Infrastructure:
- Migrate from Outlook to New Outlook (3 PCs)
- New Colour Inkjet Printer: Investigation, Purchase & Installation of Epson EcoTank
- 24Q2
- Notes:
- Status:
- 24Q1
- Notes:
- Enhance comparative Ling database - Output Vocabulary and Dialogue pages for individual languages (so that it's possible to revise them on my Iphone)
- Status:
- 23Q4
- Bridge:
- Added session-counts by partner to consolidated results
- Documentation:
- 23Q3
- Own Website
- Infrastructure:
- iCloud for Windows: Re-install & solve 'The upload folder for iCloud Photos is missing' problem. Try on new Laptop.
- Notes:
- Investigated & implemented shorter & meaningful URLs for significant Notes
- Other Websites
- Bridge:
- Read and thoroughly understand the documentation on:-
→ EBUScore
→ Bridgemate II
- 23Q2
- Own Website
- Infrastructure:
- Toshiba Laptop - Coffee over keyboard! Fixed - disabled inbuilt keboard + installed USB keyboard.
- Notes:
- Ling database: Reformated webpages for iPhone
- Enhanced comparative Ling database - Added Polish to Ukrainian vs Russian
- Other Websites
- Bridge:
- Revived Small Bridge Sites
- 23Q1
- Bridge:
- Automate NGS Data Gathering & Recalculation
- Status:
- Add a Functor to track how my daily time distribution tracks what I plan to do. Add a table of hours worked / weekday (QTD & YTD / Actual versus Plan) to Status: Summary - Actual versus Plan19.
- 22Q4
- Backups:
- De-duplicate ('prune') the backup disk. Keep a full record of deleted duplicates. One-off run.
- Infrastructure:
- Commissioning Bing with favourite selection of live own-website pages
- Researching, Purchasing & Installing Palincorp Desktop PC, Keyboard & Screen
- Migrate Windows Live Mail POP data on Toshiba to Outlook on Lenovo
- Notes:
- Reduced size of Aeon Page20 page - which had become difficult to load - by suppressing all the footnoting when a Paper had been generated (and which therefore contains the same information). Retained the link thereto.
- Amended Brief Thoughts on Language & Languages21 to have 4 priorities: Added non-Ling languages
- Spider:
- As revealed by Spider: "Aeon - Video - Organism" had author set to 'Metamorphosis' (with Note link). Maybe a manual error, combined with Espanso? Fixed 15 impacted pages Manually.
- Status:
- Bible Reading Status Reporting: Add Koran
- Fixed Year-End changes required for Oboe 'practice since last lesson'
- WebRefs:
- Hits_Pages_Total_Crosstab_Descending.htm & Hits_Bots_Total_Crosstab_Descending.htm: Pages too large to load. Fixed by:-
- Restricting page rows to total hits > 99 (this was intended, but had got lost somewhere).
- Summarising by year (rather than by month) for years before the current one.
- 22Q3
- Notes:
- Installed Espanso - and created its database: Espanso. This enables automatic addition of Note hyperlinks (without having to remember or look up the IDs) when using a text editor.
- 22Q2
- Abstracts:
- Incorrect links from Abstract to Notes (PapersToNotes Pages missing, when they should be present)
- Books:
- Created link to BooksToPapers page from banner headline in BookPaperAbstracts
- Books/Papers:
- Corrected and improved processing for Dud Abstracts Papers & Books: fixed broken links & automated process.
- Improved formatting of BookPaperAbstracts
- Notes:
- Add buttons for Lookups for Note IDs and Snippet IDs
- Add Banner Menu for links to Citations, etc, as for Papers
- Added %age read to PID Reading Lists for 'Unread' items
- Segregated PID Reading Lists - The Cut
- Fixed error in repeat citations in AddCitations_List
- Fixed bug in Reference_Notes checking for C++ false hyperlinks (C++NP string).
- WebRefs:
- Fixed Bug in Webrefs_Update where dud Wikipedia pages not recognised
- Checked whether the WebRefs checker still works now IE has been retired. It does, but I may have a problem one day when I need to install IE on a new PC.
- 22Q1
- Architecture:
- XRef-re-engineering project: Write out specific object-identifiers, and link thereto for Citations, rather than paragraph references. An issue was multiple instances of the same object in a document (fixed!).
- Notes:
- Enhanced comparative Ling database - Added Ukrainian vs Russian
- Defaulted Year & Month in Website Development Log & Oboe Reports
- Amended Brief Thoughts on Language & Languages22 to have 3 priorities
- 21Q3
- Own Website
- Notes:
- Note Snippets
- Created a method of using the same text in multiple Notes.
- Created Note_Snippets table.
- Created Functor_24, with some control over the bullets.
- Currently only used in Thesis - Chapter 01 (Introduction)23 and General Surveys24.
- May require enhancement as it gets more widely used.
- Create a page to compare the vocabulary across related languages on Ling
- Spider:
- Improved Spider performance by removing all Printable Notes - this will need to be reversed sometime.
- Status:
- Other Websites
- Bridge:
- Major Spring-clean of the Mountnessing Bridgewebs site.
- 21Q2
- Architecture:
- Created Check_Delimiters to protect against unpaired delimiters in lists, which squiff the formatting of documents that contain them. Output diagnostics to MsgBox, if not a re-gen, and to debug window to enable correction.
- Notes:
- Develop a method to create Papers for rows in Aeon_Files that have an associated Identity_Note_ID but no Paper_ID.
- Papers:
- 21Q1
- Notes:
- Enhanced Functors to work for selected non-temp Notes (ie. PID Notes: ensured Functor_22, Options 6 and 7 can run on a non-Temp Note, without creating an Archived Note; borrowed code from Auto_Reference_Notes) so that up-to-date stats can be incorporated. This was fairly complex as I wanted to avoid production of an Archived Note each time a non-Temp Note is regenerated. I also wanted to ensure that Notes whose variable text consists entirely of Functors get archived correctly (which they might not have if I'd removed Functor-generated text from Notes before saving them to the Notes table; in fact, the text saved to Live and Archive is identical and includes the full Functor content).
- Enabled multiple parameters for Functors
- Enhanced the mechanism to make the reading lists in my PID Notes easier to maintain:-
- Made the table PID_Note_Reading_Lists easier to maintain: added a menu at the start of the Create Notes Web Pages26 to (inter alia) open the PID_Note_Reading_Lists table, add or delete rows, change the "excluded" and "cut" flags, and amend Footnotes. Book/Paper titles and read-status are automatically maintained by the Functors.
- Added functionality to allow the reading lists to be further segregated - as in the Notes Animalists27 and Christian materialism28 - which din't use the Functors (though their reading lists are on PID_Note_Reading_Lists).
- Created a further Functor (Functor_23, Option 17) to generate the Thesis Chapter Reading Lists from the Notes explicitly listed as relevant in the new table Thesis_Note_XRef. Uses Functor_Indented_List, which also creates (via Functor_23, Option 16) the indented PID-Note lists. A major triumph!
- Modified Functor_21 Option 3 to allow multiple Audio Files per timesheet line
- Developed a mechanism to monitor usage of PID Notes within my Thesis (by Chapter):-
- Created Note 131729, which incorporates as table and a list, created as follows ...
- Used Notes: Functor_21, option 14 (Query PID_Notes_Used_By_Thesis_Chapter_Xtab)
- Unused Notes: Functor_22, option 5 (Query PID_Notes_Unused_By_Thesis)
- Developed a mechanism to make the reading lists in my PID Notes easier to maintain and transfer to the Chapters on my Thesis:-
- Created a new table to hold the reading lists (include some static so it's easy to maintain): PID_Note_Reading_Lists
- Created a new Functor to use this table to generate the reading lists within a PID Note (automatically segregating "read" from "unread": allowed to run with two options to allow for comments between the lists: Functor_22, Options 6 and 7, respectively).
- Enabled footnotes for reading-list items.
- Created one-off queries (PID_Notes_Books_Referenced_Gen & PID_Notes_Papers_Referenced_Gen) to extract the reading lists from PID Notes and populate the new table (using the Cross_Reference table)
- To prove the process, started manually to replace the Reading Lists in all the PID Notes with the Functors (completing this will be a manual exercise and, after each run, will involve checking Live versus Test sites).
- Procedurally, in the Thesis Chapters, if Notes are referenced in the Text, use the "NP" encoding so that they don't contaminate the reading lists: only those explicitly listed should do so.
- Added flags to either exclude Books or Papers logged in PID_Note_Reading_Lists altogether, or to segregate them as "missed the cut" in the Thesis Chapters (and, possibly) the Notes themselves.
- Automatically update table PID_Note_Reading_Lists to maintain the "read status" and Book/Paper Title.
- WebRefs:
- Webrefs_Update failed because IE looped with Aeon / Psyche pages. Performed manual checking for a while, but then Aeon seem to have fixed their pages so the problem seems to have "gone away". It doesn't occur for other pages.
- 20Q4
- Books/Papers:
- Notes:
- Spider:
- System Resources Exceeded - "Run Time Error 3035" - re-emerged in Full_Link_Same_Directory_Gen. Last month I had replaced query Full_Link_Same_Directory_Updt with identical SQL, which had worked OK. This time I removed the "WHERE" statements and put them in the code itself. Seems fine.
- Status:
- Technology:
- Consolidated Site-Hits:
- Reviewed Webalizer Methodology & Consolidated Stats
- Aggregated Robot URLs
- 20Q3
- Architecture:
- Books:
- BookCitings references not sequenced correctly. Corrected.
- BooksToNotes Link Pages: Re-engineered - used CreatePapersToNotesWebPages to generate pages
- Books/Papers:
- Notes:
- Added option in Auto_Reference_Notes to - for Live (ie. not TEMP) Notes - update the latest Archived Note to equal the updated Live Note, hence avoiding superfluous archives.
- Upgraded Auto_Reference_Notes to allow a parameterised 'termination or continue' for very long update runs
- Upgraded Auto_Reference_Notes to avoid key-words within +AA+-style Authors names (using Check_If_In_Container).
- Corrected Issues with Nested Functors: Added Functor_ID to End-tag
- Added option in Auto_Reference_Notes to only confirm new items (leaving previously-flagged items untouched)
- Highlighted Archived Notes clearly as not the latest
- Added Aeon Abstract (ie. Footnote) link to Aeon Webref items in Summary task List reports for 'read' items
- Upgraded Auto_Reference_Notes to log its actions to a new table (Auto_Reference_Notes_Actions) so that any errors can be investigated and improvements made.
- Determined why the run of Auto_Reference_Notes_Regen took 2.5 hours on 02/09/2020 (and climbing). It was down to the Aeon Note37 being searched; in particular Check_If_In_Container, checking for items in Footnotes and / or Quoted Text. As this Note is regenerated, there's no point auto-linking it. Time reduced to 23 minutes once this Note - and other Notes of inappropriate Note Groups - was removed.
- Papers:
- Process:
- Determined why Recalculation (cmdRecalculate_Click) took so long: it varied depending on how many Notes were regenerated, but seemed to be 17.5 minutes even if there are none. I think it was down to a problem with nested Functors leading to Notes growing inordinately: in particular the development logs. Now takes around 5 minutes.
- Spider:
- The size of the main database was bloating to over 1.5Gb during the spider run, so was approaching the 2Gb limit.
- Used Check_Database_Size, with a parameter, to monitor the size of the main database and added a message similar to those reporting the compact / repair of the Slave database each time a status message is printed.
- Based on the diagnostics produced, it was found that the bloating occured at two stages:-
→ In Spider_Scurry: during the creation of Raw_Links_Temp_Temp.
→ In Spider_Copy: during the creation of Raw_Links_Temp.
- Fixes:-
→ Put check in Check_Database_Size to STOP if the database is over 1.5 Gb (parameterised via Max_Database_Size).
→ Moved Raw_Links_Temp_Temp to the Slave database.
- System Resources Exceeded - "Run Time Error 3035"; query Full_Link_Same_Directory_Updt in Full_Link_Same_Directory_Gen. Fixed by using SQL rather than a query.
- Status:
- Added Weekly Project Plans to Priority Task List
- Quarterly Project Reports: Corrected Functor_08. The Project Planned YTD % kept having to be bodged!
- Created web-page (using Functor_21) showing oboe practice hours by work played
- WebRefs:
- Reformated WebLinks_Tester_Brief, WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: Allowed more space for 'link returned', 'issue' and 'display text' and added Explanation column
- Reformated WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: The 'As Above" lines waste space. Consolidate onto single second line.
- Improved WebRefs checker (Webrefs_Update) further to check for Error 403 "Forbidden". This involved finding way of checking pdfs where the returned page is in fact HTML or XML (see DevLog Ref 379).
- 20Q2
- Architecture:
- Webrefs_Update: Improved the processing of this sub which controls IE to check the URLs corresponding to WeRefs are still valid.
- Improved the recovery processing after IE fails or becomes detached. Now seems to work perfectly!
- Improved processing for 404 (page not found) errors - detect them where URL returned differs from that requested (so the error is correctly categorised) and also where it is the same (so the error is detected) - by using the GetElementsByTagName method to look through Title and H1 elemnts (where they exist).
- Note that there is still an issue for pdfs where the above methods don't work; though the page returned is an HTML or XML page, the changed URL still claims to be a pdf, so I can't check in case it's a real pdf and the process fails.
- Enhanced the Functor processing to allow parameters. Previously, I had to create a new Functor ID each time for a very similar requirement. The main problem with this old method was that these Functors (where they call another Functor, or subroutine from a row in Functor) didn't feed through into the Documenter subroutines properly.
- Truncated Cross_Reference_Changes table (from over 1M rows to under 50k) which should be self-maintaining, but isn't. This seems to have improved the regeneration times of Notes, Archived Notes and some other objects now that insertions into this table are quicker. Needs a permanent solution as part of my Cross-Referencing project.
- Authors:
- Determined why the monthly regeneration process for Authors has ballooned from under 4 hours in January 2020 to over 17 hours in April 2020. Fix on 11/04/20 - time more than halved to 8.25 hours - by re-engineering the 'menu' process that counts the items linked to further down the page.
- Enabled "alternate names" for identically-named authors (to avoid middle-initial = X):-
- Set up and populated Author_Name_Display
- Ensured Author_Name_Display appears on the Author page.
- Further reduced the monthly regeneration process time for Authors from 8.25 hours to 13 minutes! Fixed by materialising and indexing a 'view' to table Authors_Cited_By_All_List.
- Books:
- Determined why the monthly regeneration process for Book Summaries had ballooned from 1.1 hours in January 2020 to 3.5 hours in April 2020. Partly fixed by by materialising and indexing the view (now table) Book_Citings_List_New, so now takes 39 minutes. Further improvement required, given that Paper Summaries only takes 5 minutes.
- Books/Papers:
- Determined why the monthly regeneration process for Book-Paper Abstracts had doubled from 2.4 hours in January 2020 to 4.8 hours in April 2020. Successfully reduced to just upder one hour by materialising and indexing BookPaperAbstracts_List.
- Documenter:
- Notes:
- As revealed by Spider: Sundry uncategorised. Refs failing. 30 items. Fixed manually.
- Papers:
- Determined why the monthly regeneration process for Paper Summaries had ballooned from just over 3.5 hours in January 2020 to around 11.5 hours in April 2020. Fixed by materialising and indexing the view (now table) Paper_Citings_List_New, so now takes 6 minutes!
- Determined why the monthly regeneration process for Paper Abstracts had ballooned from just over 6 hours in January 2020 to over 17 hours in April 2020. Reduced to just under 4 hours as a result of materialising and indexing the view (now table) Paper_Citings_List_New. Further improvements required.
- Process:
- Wrote Check_Database_Size to check that the size of the database isn't at risk of breaching the 2GB limit. Required because of the need to re-materialise various views that have been instituted to speed up the full website regeneration routines, and which need to be run each time cmdRecalculate_Click is run, which is often.
- Full Website Regeneration took 60 hours in April 2020. This has now been substantially fixed, as of end June 2020, in that regeneration now takes just under 11 hours. While further improvements are possible, the best use of time is to avoid the need for regular regeneration altogether by completing my Cross-referencing project.
- Corrective work was split out into the items requiring improvement, must of which have now been implemented.
- History and analysis:-
- This 'balooning' was on my new laptop which has solid state disks - it was taking 36 hours on my old laptop, then halved when I got the replacement, but then nearly doubled!
- I investigated why this was so, and improved performance mainly by putting in a trace (using a timer - GetTickCount - found on the web that allows logging of elapsed time in milliseconds, using StartTimer and EndTimer) and finding the 'pinch points'
- I suspected that it was a new release of MS Access that's caused a problem; maybe a lost index or something similar. I couldn't find any evidence of this, but a mumber of queries were taking a second or so to execute, which small times multiplied up drastically when the number of pages on my site - over 100k - are taken into account. Quite why a cliff-edge had been reached, I know not!
- Three areas had been particularly impacted: Authors, Paper Abstracts & Paper Summaries, which have all trippled or worse. These are covered as seperate developments under 'Authors' and 'Papers'.
- Book Summaries and Book-Paper Abstracts had also at least doubled, but this has had less of an impact as they took much less time in the first place.
- Other regeneration processes - in particular 'Notes' - didn't appear to have been impacted.
- WebRefs:
- WebLinks_Tester_Full & WebLinks_Tester_Full_Map: Reformated Jump Table to 30 columns and multiple rows.
- As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 184 items. res://ieframe.dll/ in Returned_URL.
- Amend WebRef 'Name' link to #Off-Page_Link_WxxxW style. Also amend Quarterly and Annual task lists to use this reference.
- WebLinks_Tester: If the URLs of WebRefs are changed between Spider runs, the URL mapping fails, and the WebRefs appear at the end of the last page, wrongly categorised. Fixed to ensure they appear on their own page, added to the jump table as 'WebRef Missing' and added an explanation.
- Amended WebRef 'Name' links to #Off-Page_Link_WxxxW style so referencing from other pages is possible (useful for Aeon)
- 20Q1
- Abstracts:
- As revealed by Spider: Link to PaperCitings failing (2 items) - Timing issue
- Authors:
- As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items. Dud page (Ontological Argument) deleted.
- Restrict jump-table at top of Authors pages to relevant items, and include the item counts.
- Bridge:
- As revealed by Spider: Bridge_Code_Archive.htm (14 items) & Bridge_Code_Archive_Recent.htm (5 items)
- Notes:
- As revealed by Spider: Blackberry links from archived Note 1010 (Maintenance Dashboard). 38 items.
- Solve the problem of the size limitation for Notes by allowing their concatenation.
- As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
- As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
- As revealed by Spider: Blog39 Achived Notes - sundry links to Printable Notes. Live Note OK. 97 items.
- Papers:
- Make use of Comments for large Abstracts seamless by appending Comments (if > 5k) to Abstracts.
- WebRefs:
- As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 51 items outstanding. res://ieframe.dll/ in Returned_URL.
- Look into Sistrix Smart41. Errors and warnings itemised are:-
- Duplicate content: seems to be variants on theotodman.com
- Title Tags: Empty, too long, identical
- Page Not Found
- Filesize in excess of 1Mb
- Meta-Description: Empty
- Few words on Page
- H1: Not used, used multiple times per page, identical across pages
- Pictures: Alt attribute missing
Since at least August 2022, Sistrix has only produced 3 errors: home page not accessible, therefore no other pages accessible and also not HTTPS.
- As revealed by Spider: WebLinks_Tester_4.htm. Refs failing. 32 items. res://ieframe.dll/ in Returned_URL.
- WebRefs checker failing to detect broken links for Aeon - if a link fails, I get the message 'The page you were looking for could not be found (404)', but since the URL doesn't change, I don't spot it. This may happen for other sites as well. Fixed for Aeon - watch out for others.
- 19Q4
- Animadversions:
- Create Aeon_Files table & code to generate Aeon Page40 (to simplify processing and get round the 64k character limit).
- Authors:
- Improve author-citations process:-
- Show author-citation counts on Author Summary pages.
- Only have an author-citation link on Authors pages if the author is cited!
- Import Authors from Aeon_Files table.
- Notes:
- Create procedures (manual and via Functor_17) to add Audio Files to Notes
- Process:
- Fix Functors to enable previous quarter's Status Reports after Time period roll-over
- 19Q3
- Own Website
- Authors:
- As revealed by Spider: Author pages not regenerated when name corrected, leading to Sundry broken Links from other pages.
- Books/Papers:
- Print Write-up Notes (not just the links) for appropriate Papers in BookPaperAbstracts
- Bridge:
- NGS Refinements
- Synchronise Documentator with main website generator
- Add Mountnessing archive links to Results
- Infrastructure:
- PDFelement: Purchase & Commission upgrade from Version 5 to Version 7 for $39 - await new Laptop
- Purchase & Commission new Lenovo Laptop: awaiting replacement following motherboard failure!
- WebRefs:
- As revealed by Spider: Hits_Pages_Total_Crosstab_Descending.htm.
- Other Websites
- Sophie:
- 19Q2
- Authors:
- Added Jump Table to sections of Authors Pages
- General:
- Created 'Chess' Home Page
- Created as a main menu item and Note42
- Created consolidated results pages
- Added games & links to pgn4web
- Status:
- Corrected Quarterly Reporting for uneven Plan (Holidays)
- 19Q1
- Books/Papers:
- Removed Blackberry processing and associated pages as no longer needed with fast broadband.
- Photos:
- Created Timeline software
- Created Photo-logging usage via Images table.
- Timeline software: Added photos of Coxes Farm
- Technology:
- Investigated the Hit Stats provided by Webalizer and produced web-pages to show the most active pages.
- 18Q4
- Own Website
- Architecture:
- As revealed by Spider: Old links in unregenerated pages. 8 Items.
- Documenter:
- Outstanding Developments by Category appearing all in Bold, and links to lists failing.
- Notes:
- As revealed by Spider: Links from Blog in Level 1 or 2 Printable Notes 51243 and 98144 failing. 141 items.
- Create Priority Task List Report45.
- As revealed by Spider: Links to sundry 'static' pages failing.
- 13 items fixed.
- 7 items outstanding: to do with Pete's PhD Test site. Links seem to be OK, so may be a problem with the Spider. Directory "Petes_PhD" was the only one set to "do not parse". The file wasn't in the site map. Reset; hopefully it'll work!
- As revealed by Spider: Sundry links failed for Supervisions. 30 items.
- As revealed by Spider: Status Reports - Achived Notes - sundry links failing. 50 items in total. Live Notes fixed.
- Website: Maintenance Dashboard46. 14 items.
- Personal Identity Status Report47. 11 items.
- Summary Status Report48. 4 items.
- Actual Detail Summary Report49. 6 items.
- Chess Status Report50. 12 items.
- Etc. 3 items.
- As revealed by Spider: Consciousness Status Report51 Achived Notes - links to BookCatalogCategorised_16_115_471.htm failing. Live Note OK. 46 items. 24 items outstanding.
- Process:
- Spider:
- Determined why Full_Link_Up_Levels_Gen takes so long - approximately 4 hours. Re-architected to reduce database reads.
- Status:
- WebRefs:
- Added Display text to WebRefs for Links to Stanford, Wikipedia & Aeon
- Other Websites
- Enigma:
- Added 'EE' shortcut icon on all Webpage browser tabs
- Sophie:
- Added 'SB' shortcut icon on all Webpage browser tabs
- 18Q3
- Architecture:
- Added 'TT' shortcut icon on all Webpage browser tabs
- Authors:
- As revealed by Spider: 'Extra' Links to Notes from Citations List have the wrong directory (clumped in 1,000's like Papers rather than 100's). 7 items. Problem with AddCitations_List, so not just an issue for Authors.
- Authors Narrative: enable footnotes.
- Books:
- As revealed by Spider: BookCitings. 405 items. Fixed problem with CreateBookCitingsWebPages not checking for change of Book during 'Repeats' processing.
- Books/Papers:
- Paper & Book Abstracts - Make 'Disclaimer' more prominent & show 'Paper Statistics' rather than 'Paper Summaries'
- Bridge:
- Reformated + Fixed Hand-dealing pages to CSS standards
- Documenter:
- Created Functor using Form_Documentation_Links to automatically add control-links to documentation Notes.
- Fixed Bug where a variable has the same name as a procedure, and is thereby confused with one. Produced a warning query (Variable_Code_Name_Clashes) for manual correction.
- Footnotes:
- Notes:
- As revealed by Spider: Languages Status Report55 Achived Notes - links to Dissertation failing. Live Note OK. 43 items.
- As revealed by Spider: Notes_List_Control links fail for Supervisions. 58 items.
- Automatic Key-word Hyperlinking to Notes Project: Phase 2
- Added links based on Book & Paper Titles
- Added ***Read***-marker to Lists
- Allowed links to be ignored (rather than suppressed) during approval process
- Skip links to Status Reports, etc.
- Created new '+LL+' Functor for links to Note_Link pages; fixed associated Notes.
- Blog: Created Functor to generate Blog from Blog.
- As revealed by Spider: Links to Notes_xxx_Links, from Abstracts (94 items), Notes (25 … corrected manually), NotesPrint (1916). Fixed by +LL+ functor implementation.
- Created auto-regen of all Note Link (Note_nnn_Link) Pages
- As revealed by Spider: Religion Status Report58 Achived Notes - sundry links failing. Live Note OK. 59 items.
- As revealed by Spider: Bridge Status Report59 Achived Notes - sundry links failing. Live Note OK. 70 items.
- Photos:
- Status:
- 18Q2
- Architecture:
- Adjusted MS Access Control Page62 to feature new functions (Development Log & Automatic XRef) and consolidated the categorised Notes Lists into one button.
- Documenter:
- Investigated the error reports from the Documenter:
- Improved the diagostics
- Removed unused variables from the Generator code.
- General:
- Website Tidy-up:-
- Wrote a function to delete - with specified exceptions - items not regenerated in the last full website regeneration.
- Tidied up Site - deleted non-updated pages (especially old Printable Notes). Automatically on the local site, manually - using the audit log - on the live site.
- Investigated setting up an FTP link as a drive so I could automatically delete from the live site, but couldn't get it to work. Easy to set up as a drive, but not to assign a drive-letter so the drive can be used by VBA.
- Notes:
- Added a process to 'fix' - by adding the Note-ID - those note-links added without stating the ID. This will save processing time.
- Created automatic key-word referencing (ie. to Notes in the Notes-Group "Personal Identity"). This was complicated for a number of reasons:-
- There are always false positives, so there needed to be a way of "proofing" what's to be done before doing anything.
- There will be times where it is inappropriate (eg. in quoted work by other authors). So, there had to be a way of permanently suppressing such links.
- The referencing differs depending on whether the term is in a Footnote or not.
- I needed to check for already-existing links, variant grammatical forms, etc.
- I created a MsgBox routine - providing context - so that what needed to be done with potential links could easily be decided when the linking process was (re-)run.
- Links are always to Notes, but are from Notes, Archived Notes, Authors and Book & Paper Abstracts & Comments.
- Papers:
- Papers associated with broken links:-
- Created an area to hold pdfs, and automated links thereto.
- Created a pop-up to say I'd remove such pdfs if requested by their authors.
- Consequently, haven't password protected the area yet.
- Process:
- Re-engineered Website Maintenance Dashboard63 to show extra topical links (and show them at the top of the page).
- Status:
- Added extra items to Functors to facilitate Quarterly Reporting:-
- Automatically calculated QTD & YTD time statistics in the quarterly reports.
- Provided a YTD task list for Status Reports for projects "in abeyance" to make reporting easier.
- Modified the statistics - to be just YTD - for YTD Status reports.
- Created table for - by Project - "Plans for the near future" tasks, and a functor to add them to the status report.
- Took "items competed" and "plans for the near future from the development log for the Web-tools status report.
- Created another functor to add these lists to the Summary report.
- WebRefs:
- 18Q1
- Bridge:
- Corrected format of Links from Bridge Page
- Documenter:
- Documenter Bug-fixes: Objects in Comments, MsgBoxs & Debugs
- Created consolidated Development Log, and updated "Progress to Date" and "Outstanding Developments" Reports
- Infrastructure:
- Migration of Website from UKServers to Namesco. Test out.
- Notes:
- Papers:
- Correction of narratives on Note time-recording pseudo-papers
- Process:
- Wrote a routine to update a set of .htm files, replacing text in them all
- Improved efficiency of daily changes regeneration (removed msgboxes)
- Status:
- Created web-page generator for Development items (ie. Created the Web-Project "To Do" List and "Outstanding Developments" directly from this list).
- WebRefs:
- Documentation & Bug-fixes: Phase 1
- Completed "as is" functional documentation so I could see what's going on.
- Re-engineered so it made sense and ran efficiently without bugs
- Fixed the obvious data errors that confused the statistics - ie. where the returned URL differs from that requested and https issues.
- Wrote / improved diagnostic reports (web pages)
- Determined & reported usage directly using embedded +WnnnW+ encoding rather than via the Spider. Reconciled the two methods.
- 17Q4
- Architecture:
- Automated links between Quarterly reports (Task Lists)
- Books/Papers:
- New "Recent Acquisitions" Book List - Uncategorised
- Documenter:
- Code Archive: Created Web-pages of Recent Code Changes
- Code Archive: Hadn't been working - fixed and recovered a selection from backups
- Notes:
- Added %age complete to Notes Reading Lists
- Enabled deduction of Note link by omitting the Note ID from the mark-up and doing a look up on the key-word and its varients.
- Timesheets:
- Separated Time Recording for Aeon
- 17Q3
- Notes:
- Added Colour-coded "quality markers" to Notes: included them in lists and jump-tables
- 17Q2
- Bridge:
- Created hyperlinks from my consolidated results pages to the actual results
- Search:
- Wrote routines to create a cut-down Website excluding pages with irrelevant or duplicated information. Used this to create the ZoomSearch facility.
- 17Q1
- Bridge:
- Created Master Point Lists & Promotions for the ECBA website.
- Added new features (70s Club & Monthly Pairs Winners) to my Bridge mini-websites.
- 16Q4
- Spider:
- Added “.shtml” and “.shtm” to the list of web-suffices.
- 16Q3
- Acts:
- Developed a website for displaying the textual and grammatical analyses and appendices of Pete’s PhD on the Acts of the Apostles. Two versions: Live (Acts: Live Site (Defunct)) and Test (Acts: Test Site).
- Bridge:
- 16Q2
- Clock:
- Created a "chess clock" application to help time tasks. Not much use in the end.
- 16Q1
- Own Website
- Books/Papers:
- Created direct links from Subject, Topic & Sub-topic (in Book and Paper Summaries) to their link pages.
- Documenter:
- Created an automatic Forms documenter – with links to the code.
- Notes:
- Stopped (I hope!) “private” Notes showing in Paper Abstracts.
- Enabled direct links to Notes from within Footnotes.
- Allowed Note- & Footnote-links from multiple words73. This is such a useful feature!
- Completed converting “static” pages to Notes.
- WebRefs:
- Some tidying up of the 3,000-odd external links from my website.
- Other Websites
- Bridge:
- Created Ladies’ and Gentlemen’s ladders for my mini-websites.
- Reconciled my X-IMP calculations with a new feature provided by Bridgewebs.
- Accommodated Alaric’s new scoring system for St. Edith’s.
- 15Q4
- Own Website
- Abstracts:
- Corrected data format on Identity Abstracts
- Aesthetics:
- Reduced font + changed text in ubiquitous THEO'S PAGE heading
- Authors:
- Corrected author's names (SN (FN)) for all papers and books
- Provided easy links to works by the same author; Books, Papers; Joint authors
- Backups:
- Enhanced my backup system to prune items backed up multiple times (eg. if they moved directory).
- Books/Papers:
- Neatened up the linkage headers in Book & Paper Abstracts.
- Updated booklists to include shelf-identifiers
- Data:
- Corrected data content on Identity Sub-topics
- Documenter:
- Segregated the code-archive, which was bloating74 the generator database.
- Notes:
- Converted the “static” pages in my website to Note format (so they can change and retain history).
- Fixed problem of images not appearing in Printable Notes (wrong directory level) unless the images are repeated
- Process:
- Created a consolidated processor to enable overnight regeneration of the whole website.
- Spider:
- Finally cracked the Spider for external links, and used it to check the 3,000-odd links from my website. Some tidying up outstanding.
- Other Websites
- Bridge:
- Segregated the Bridge website generators from those for my own website. They now live in separate databases and have their own documenters.
- 15Q3
- Own Website
- Architecture:
- Made a lot of progress on improving the hyperlinking in my own website – I’ve made a good start on this project. The links now work accurately, but I need to ensure the efficient propagation of linkage changes across the site without requiring a complete regeneration.
- Notes:
- Tidied up the archived Notes.
- Other Websites
- Bridge:
- Ported my “handicap / improvers” routines from Hutton to “Mini Websites”.
- Created a new website (using Bridgewebs) for the First Class Bridge Academy (First Class Bridge).
- Enigma:
- Created a small website for a music group Julie and I attend – the Enigma Ensemble.
- Sophie:
- 15Q2
- Bridge:
- Development of the “Eve Goblets” (victor ludorum) software for the ECBA, and roll-over to the new season.
- 15Q1
- Own Website
- Architecture:
- Vastly improved the “bulleting” options in Notes, etc.
- Authors:
- Improved the format and content of “Authors” pages.
- Timesheets:
- Segregated time-recording so I can easily see how much time is spent developing and supporting my own website, and how much is spent on those of other organisations.
- Other Websites
- Bridge:
- Started a project to provide small websites for bridge clubs without sites: Essex Bridge Results.
- Took on the ECBA Website (ECBA Website (Defunct)), at the cost of considerable effort by way of reformatting & tidy-up.
- Further improvements to my Butler-IMPs and Cross-IMPs calculation routines.
- 14Q4
- Own Website
- Notes:
- Made improvements to the regeneration and referencing of archived Notes.
- Spider:
- Fixed the bug wherein the Spider ran out of system resources on large runs: now compacts & repairs the slave database as it does along (every 200k links; parameterised).
- Other Websites
- Bridge:
- Improvements to my Butler-IMPs and Cross-IMPs calculation routines – different formats, including the Jeff Smith format, and extension to Butler-scored Pairs events. This work has been taken up by the ECBA selectors for county-team selection.
- 14Q3
- Own Website
- Notes:
- Started to split Paper Abstracts from Write-ups.
- Created a progress-tracker72 for my Thesis.
- Improved error-recovery where encoding fails for Notes links.
- Spider:
- Continued working on my Spider and started to use it to clean up my website by diagnosing broken links.
- Other Websites
- Bridge:
- Created a suite of programs to extract data and calculate Butler-IMPs and Cross-IMPs from the travellers on Bridgewebs Teams events, and output ranking-lists to the Web.
- 14Q2
- Architecture:
- Added extra linkages between pages (eg. on Author).
- Authors:
- Created the facility for Author Narratives.
- Backups:
- Used a clone of my Spider routines to create a backup and restore system.
- Documenter:
- Created multiple secure areas on my website, so that Documentation is no longer hiding under my Supervision Notes.
- Notes:
- Wrote routines to import old (but missing) archived Notes into the database so they can be (and have been) re-exported in the current format.
- 14Q1
- Architecture:
- Repaired / updated my Website site-map
- Notes:
- Made a lot of improvements to my Site generators, in particular the pesky bug that stopped the links to archived Notes working correctly.
- Process:
- Improved the regeneration routines for the various classes of webpage, and got them to record statistics in support of the “Website Regeneration Dashboard”.
- Created a “Website Regeneration Dashboard” to show when the various elements of my website have been refreshed, and how long the routines took to run. Links to this appear in the footer to each webpage.
- 13Q3
- Architecture:
- Sorted conflicts between Subject, Topic & Sub-Topic IDs
- Books/Papers:
- Made enhancements to "Books & Papers Collection Categorised" Pages
- Improved linkages between Book/Paper Lists and Summaries or Abstracts
- Notes:
- Simplified / Re-engineered Notes processing
- "Single Image" development for Notes
- Allowed "Authors Works Lists" to be linked-to from Notes
- Created Archived Notes Re-Export facility
- Status:
- Re-engineered Monthly Reporting on Change of Year
- Technology:
- Used CSS in all manually-crafted and automatically-generated pages
- Converted Tables to CSS3 standard
- Timesheets:
- Re-engineered Time-Recording, so that actuals are held against Papers (and Books) in a separate table, rather than as columns by year in the Papers / Books tables themselves. Retained the grand totals only.
- 13Q2
- Own Website
- Architecture:
- Improved error-recovery where encoding fails for links to Books, Papers and Authors. That is, where one of my bespoke tags (eg. +B77) doesn’t match with its corresponding anti-tag (eg. B+).
- Replaced macros with code.
- Books/Papers:
- Developed "Books by Location" Pages
- Improved the linkage between Papers and Books.
- Bridge:
- Created “Consolidated Bridge Results” webpages (for my own results).
- Documenter:
- Wrote the Code Documenter.
- Created a code & query archive facility as part of the Code Documenter.
- Notes:
- Improved "Concatenated Notes" production
- Photos:
- Reduced the size of photos (to make the pages quicker to load).
- Spider:
- Status:
- Improved Quarterly Reporting
- Improved "This Month's Summary" Reporting
- WebRefs:
- Some pages render oddly in Chrome (and maybe other browsers) – investigated78.
- Other Websites
- Bridge:
- Merged79 Philosophy & Bridge Databases
- 13Q1
- Status:
- Re-engineered Monthly Reporting Tables & improved Monthly Reporting
- Technology:
- 12Q3
- General:
- Improved Titles for Search Results display optimisation
- 11Q4
- Books/Papers:
- Showed Percentage of Paper / Book Read
- 11Q2
- Authors:
- Created Author Cross-reference
- Books/Papers:
- Created Inter-Book-Paper Citings Cross-reference
- Added “changed dates” to Paper and Book abstracts, and produced “recently changed” pages for them
- 10Q3
- Notes:
- Wrote routines to automatically generate the “Printable Notes” footer – and to create the corresponding web pages – from the control table. Very useful.
- Add “Date Created” to Note_Links (for audit purposes; I’m still not convinced that links disappear when they should).
- Investigate Note_Links: needs versioning, and section of calling Note
- WebRefs:
- Improved the functionality for automatically generating external Web-links in Notes, Book Abstracts and Paper Abstracts. Corrected current data to accommodate the new code.
- 10Q2
- Footnotes:
- Added intra-Note footnotes – a real breakthrough (strange I’d not thought of this before). It will enable me to convert to Note form all the old .pdfs from my undergraduate days – which are replete with footnotes – if this is any use.
- Spider:
- Wrote my own Web Spider.
- It’s a little bit of a cheat, in that it doesn’t actually invoke the web-links, but crawls through my directory structure selecting all the .htm files, and rummages through the HTML looking for links which it then copies to a table.
- It also maintains a table of the directory structure and files therein, and then works out the real address from the relative addressing.
- It then allows cross-checking between the links and the files.
- Links external to the site have to be investigated manually (at least I have a list of all the links) – unless I can work out how to use the list to power a browser to check.
- There are about 333k links in 55k files – the full run took about 8 hours.
- It now runs incrementally based on file last-updated dates.
- 09Q4
- Notes:
- Restricted the size of the “previous versions” tables for Notes to 12.
- Process:
- Set up my database system to roll forward to the next academic year, and generate web-pages appropriately.
- 09Q3
- Books/Papers:
- Completed a full 2-way link from Book/Paper Summaries to Notes.
- Made a lot of changes to the way Books and Papers are categorised; in particular:-
- Added 2-level jump tables, to save the browser being crowded with hundreds of cross-categorisations in the same page.
- While it’s OK for Books, it seems to have a few potholes for Papers, as some of the links fail.
- The problem may be something to do with my economical way of suppressing redundant sub-categorisation - ie. where one or more of Subject, Topic and Sub-topic are the same.
- Notes:
- Functionally enhanced the Notes-generation processing options; to make niftier production possible. In particular, where individual Notes are selected, I made the re-generation of the cross-references optional (so that if only textual variation is being iterated, the iterations are quicker).
- Improved the performance of “All Notes” regeneration. The problem was with the generation of the generic reading lists based on Note Title. These only apply to the “Identity” Notes, and it was taking a long time to determine there was no list to produce.
- Ensured that the Notes Reading-list is generated from all 3 sub-topics.
- Enabled Concatenated Notes (by Group). This is only allowed for “Identity” Notes at the moment, which is what I’d wanted it for. It allows all Notes for a Group to be printed off in Title sequence, for ease of review.
- Enabled automatic immediate promotion to Live Website of selected Temp Notes.
- Process:
- Created a “performance stats” database and routines to write to it – though I only implemented them for the Notes generator.
- 09Q2
- Architecture:
- Created Site-map for Website
- Books/Papers:
- Improved, and corrected errors in, the Blackberry-friendly formatted lists of Books and Papers.
- Added "quality marker" to Paper & Book Abstracts
- De-duplicated Books / Papers to Notes links
- Notes:
- Added a table of Notes history at the end of each Note (ie. not just previous version) - to save tedious chaining
- 09Q1
- Books/Papers:
- Provided an automated link from Books / Papers to the Notes referencing them
- Improved the Books / Papers to Notes links to include the right Version and Section (and historical Title) of the Note
- Enhanced Books to Papers links to include papers multiply held
- Created automatic links from Paper Summary to Book Summary
- Enabled links from Books to the Papers contained therein
- Notes:
- Added reading-list (of works cited) at the end of the Note, as well as in the Printable Note
- Fixed the problem of the overpopulation of the Notes directory
- Option for Notes-Regen:
- Changed-only
- Changed, and all impacted by Change
- All
- Created optional "don't print" feature for specific footnotes.
- 08Q4
- Timesheets:
- Updated Book totals from totals of the Papers contained therein
- 08Q1
- Notes:
- Allowed regeneration of Temp Notes only
- 07Q4
- Architecture:
- Added password-protection to selected pages
- Notes:
- Added referenced reading lists to printable version of Notes
- Improve "Temp" note processing. Suppress "_T" suffix (just don't copy to Transfer)
- When a Note is archived, ensured a printable note is also archived81 (if this Note has a print)
- Added referenced Books / Papers as footnote jump table to Notes
- Search:
- Created search facility using ZoomSearch
- 07Q3
- Animadversions:
- Added a "hot topics" link - things that I'm currently working on and want feedback ("Animadversions")
- Footnotes:
- Improved transparency of Footnote referencing in Printable Notes
- Notes:
- Made Note-change-history available on-line (added status + publishable flags)
- Created82 a "quick lookup" table of responses to a note and its sub-notes (recursive function on Notes_Links)
- Created "respondent notes" - database identifier & red indicator + text
- Enabled generation of printable version of Notes
- 07Q2
- Abstracts:
- Segregated Abstracts into multiple directories
- Created auto-backup of Abstracts, and more efficient maintenance web-routines
- Added "Extra Data" parallel to Abstracts; ie. "More …" (Location etc.)
- Architecture:
- Tidied up Footers throughout Philosophy Papers site (made into tables)
- Segregated database83 into Master and Slaves (code, papers, books, notes, web control, … vs PI)
- Books/Papers:
- Added "read" column in papers / books lists
- Created "Papers by Title within Topic" list
- Created "Identity Read Papers by Sub-Topic" list
- Added Flags for "laptop" and "photocopy"; removed document location from Comments
- Added Book Comments and Abstracts (as "More…") to Book-lists
- Added multiple Jump-levels for BB Lists (to keep pages small)
- Added change of author or title in "Abstracts audit"
- Created "Identity Papers Full by Sub-Topic" list
- Jump-Tables: added "by topic" option
- Created "BB-friendly84" lists: Papers (Full) and Books
- Footnotes:
- Created auto-backup of Footnotes85, and more efficient maintenance web-routines
- Created auto-renumbering86 routine for Footnotes
- General:
- Improved "Disclaimer87" Page. Reduce font, add ref to philosophy databases + hyperlinks
- Notes:
- Allowed two sorts of Note - document-specific ("Write-up") and general
- Added "outgoing links" summary footer to Notes pages
- Added "footnote" pages88 for Carthusians & King's
- Process:
- Improved ease of FTP process by use of Transfer Directory
- WebRefs:
- Made Web Refs in Comments / Abstracts into Hyperlinks
Items Completed by Category:
- Own Website: Completed Items By Category:-
- Abstracts
- 22Q2
- Incorrect links from Abstract to Notes (PapersToNotes Pages missing, when they should be present)
- 20Q1
- As revealed by Spider: Link to PaperCitings failing (2 items) - Timing issue
- 15Q4
- Corrected data format on Identity Abstracts
- 07Q2
- Added "Extra Data" parallel to Abstracts; ie. "More …" (Location etc.)
- Created auto-backup of Abstracts, and more efficient maintenance web-routines
- Segregated Abstracts into multiple directories
- Aesthetics
- 15Q4
- Reduced font + changed text in ubiquitous THEO'S PAGE heading
- Animadversions
- 19Q4
- Create Aeon_Files table & code to generate Aeon Page76 (to simplify processing and get round the 64k character limit).
- 07Q3
- Added a "hot topics" link - things that I'm currently working on and want feedback ("Animadversions")
- Architecture
- 22Q1
- XRef-re-engineering project: Write out specific object-identifiers, and link thereto for Citations, rather than paragraph references. An issue was multiple instances of the same object in a document (fixed!).
- 21Q2
- Created Check_Delimiters to protect against unpaired delimiters in lists, which squiff the formatting of documents that contain them. Output diagnostics to MsgBox, if not a re-gen, and to debug window to enable correction.
- 20Q3
- 20Q2
- Webrefs_Update: Improved the processing of this sub which controls IE to check the URLs corresponding to WeRefs are still valid.
- Improved the recovery processing after IE fails or becomes detached. Now seems to work perfectly!
- Improved processing for 404 (page not found) errors - detect them where URL returned differs from that requested (so the error is correctly categorised) and also where it is the same (so the error is detected) - by using the GetElementsByTagName method to look through Title and H1 elemnts (where they exist).
- Note that there is still an issue for pdfs where the above methods don't work; though the page returned is an HTML or XML page, the changed URL still claims to be a pdf, so I can't check in case it's a real pdf and the process fails.
- Enhanced the Functor processing to allow parameters. Previously, I had to create a new Functor ID each time for a very similar requirement. The main problem with this old method was that these Functors (where they call another Functor, or subroutine from a row in Functor) didn't feed through into the Documenter subroutines properly.
- Truncated Cross_Reference_Changes table (from over 1M rows to under 50k) which should be self-maintaining, but isn't. This seems to have improved the regeneration times of Notes, Archived Notes and some other objects now that insertions into this table are quicker. Needs a permanent solution as part of my Cross-Referencing project.
- 18Q4
- As revealed by Spider: Old links in unregenerated pages. 8 Items.
- 18Q3
- Added 'TT' shortcut icon on all Webpage browser tabs
- 18Q2
- Adjusted MS Access Control Page89 to feature new functions (Development Log & Automatic XRef) and consolidated the categorised Notes Lists into one button.
- 17Q4
- Automated links between Quarterly reports (Task Lists)
- 15Q3
- Made a lot of progress on improving the hyperlinking in my own website – I’ve made a good start on this project. The links now work accurately, but I need to ensure the efficient propagation of linkage changes across the site without requiring a complete regeneration.
- 15Q1
- Vastly improved the “bulleting” options in Notes, etc.
- 14Q2
- Added extra linkages between pages (eg. on Author).
- 14Q1
- Repaired / updated my Website site-map
- 13Q3
- Sorted conflicts between Subject, Topic & Sub-Topic IDs
- 13Q2
- Improved error-recovery where encoding fails for links to Books, Papers and Authors. That is, where one of my bespoke tags (eg. +B91) doesn’t match with its corresponding anti-tag (eg. B+).
- Replaced macros with code.
- 09Q2
- Created Site-map for Website
- 07Q4
- Added password-protection to selected pages
- 07Q2
- Segregated database92 into Master and Slaves (code, papers, books, notes, web control, … vs PI)
- Tidied up Footers throughout Philosophy Papers site (made into tables)
- Authors
- 20Q2
- Determined why the monthly regeneration process for Authors has ballooned from under 4 hours in January 2020 to over 17 hours in April 2020. Fix on 11/04/20 - time more than halved to 8.25 hours - by re-engineering the 'menu' process that counts the items linked to further down the page.
- Enabled "alternate names" for identically-named authors (to avoid middle-initial = X):-
- Set up and populated Author_Name_Display
- Ensured Author_Name_Display appears on the Author page.
- Further reduced the monthly regeneration process time for Authors from 8.25 hours to 13 minutes! Fixed by materialising and indexing a 'view' to table Authors_Cited_By_All_List.
- 20Q1
- As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items. Dud page (Ontological Argument) deleted.
- Restrict jump-table at top of Authors pages to relevant items, and include the item counts.
- 19Q4
- Improve author-citations process:-
- Show author-citation counts on Author Summary pages.
- Only have an author-citation link on Authors pages if the author is cited!
- Import Authors from Aeon_Files table.
- 19Q3
- As revealed by Spider: Author pages not regenerated when name corrected, leading to Sundry broken Links from other pages.
- 19Q2
- Added Jump Table to sections of Authors Pages
- 18Q3
- As revealed by Spider: 'Extra' Links to Notes from Citations List have the wrong directory (clumped in 1,000's like Papers rather than 100's). 7 items. Problem with AddCitations_List, so not just an issue for Authors.
- Authors Narrative: enable footnotes.
- 15Q4
- Corrected author's names (SN (FN)) for all papers and books
- Provided easy links to works by the same author; Books, Papers; Joint authors
- 15Q1
- Improved the format and content of “Authors” pages.
- 14Q2
- Created the facility for Author Narratives.
- 11Q2
- Created Author Cross-reference
- Backups
- 24Q4
- Purchase & commission new 5Tb WD Backup Drive. Copy 2Tb backup drive contents across.
- 22Q4
- De-duplicate ('prune') the backup disk. Keep a full record of deleted duplicates. One-off run.
- 15Q4
- Enhanced my backup system to prune items backed up multiple times (eg. if they moved directory).
- 14Q2
- Used a clone of my Spider routines to create a backup and restore system.
- Books
- 22Q2
- Created link to BooksToPapers page from banner headline in BookPaperAbstracts
- 20Q3
- BookCitings references not sequenced correctly. Corrected.
- BooksToNotes Link Pages: Re-engineered - used CreatePapersToNotesWebPages to generate pages
- 20Q2
- Determined why the monthly regeneration process for Book Summaries had ballooned from 1.1 hours in January 2020 to 3.5 hours in April 2020. Partly fixed by by materialising and indexing the view (now table) Book_Citings_List_New, so now takes 39 minutes. Further improvement required, given that Paper Summaries only takes 5 minutes.
- 18Q3
- As revealed by Spider: BookCitings. 405 items. Fixed problem with CreateBookCitingsWebPages not checking for change of Book during 'Repeats' processing.
- Books/Papers
- 24Q4
- 22Q2
- Corrected and improved processing for Dud Abstracts Papers & Books: fixed broken links & automated process.
- Improved formatting of BookPaperAbstracts
- 20Q4
- 20Q3
- 20Q2
- Determined why the monthly regeneration process for Book-Paper Abstracts had doubled from 2.4 hours in January 2020 to 4.8 hours in April 2020. Successfully reduced to just upder one hour by materialising and indexing BookPaperAbstracts_List.
- 19Q3
- Print Write-up Notes (not just the links) for appropriate Papers in BookPaperAbstracts
- 19Q1
- Removed Blackberry processing and associated pages as no longer needed with fast broadband.
- 18Q3
- Paper & Book Abstracts - Make 'Disclaimer' more prominent & show 'Paper Statistics' rather than 'Paper Summaries'
- 17Q4
- New "Recent Acquisitions" Book List - Uncategorised
- 16Q1
- Created direct links from Subject, Topic & Sub-topic (in Book and Paper Summaries) to their link pages.
- 15Q4
- Neatened up the linkage headers in Book & Paper Abstracts.
- Updated booklists to include shelf-identifiers
- 13Q3
- Improved linkages between Book/Paper Lists and Summaries or Abstracts
- Made enhancements to "Books & Papers Collection Categorised" Pages
- 13Q2
- Developed "Books by Location" Pages
- Improved the linkage between Papers and Books.
- 11Q4
- Showed Percentage of Paper / Book Read
- 11Q2
- Added “changed dates” to Paper and Book abstracts, and produced “recently changed” pages for them
- Created Inter-Book-Paper Citings Cross-reference
- 09Q3
- Completed a full 2-way link from Book/Paper Summaries to Notes.
- Made a lot of changes to the way Books and Papers are categorised; in particular:-
- Added 2-level jump tables, to save the browser being crowded with hundreds of cross-categorisations in the same page.
- While it’s OK for Books, it seems to have a few potholes for Papers, as some of the links fail.
- The problem may be something to do with my economical way of suppressing redundant sub-categorisation - ie. where one or more of Subject, Topic and Sub-topic are the same.
- 09Q2
- Added "quality marker" to Paper & Book Abstracts
- De-duplicated Books / Papers to Notes links
- Improved, and corrected errors in, the Blackberry-friendly formatted lists of Books and Papers.
- 09Q1
- Created automatic links from Paper Summary to Book Summary
- Enabled links from Books to the Papers contained therein
- Enhanced Books to Papers links to include papers multiply held
- Improved the Books / Papers to Notes links to include the right Version and Section (and historical Title) of the Note
- Provided an automated link from Books / Papers to the Notes referencing them
- 07Q2
- Added "read" column in papers / books lists
- Added Book Comments and Abstracts (as "More…") to Book-lists
- Added change of author or title in "Abstracts audit"
- Added Flags for "laptop" and "photocopy"; removed document location from Comments
- Added multiple Jump-levels for BB Lists (to keep pages small)
- Created "BB-friendly94" lists: Papers (Full) and Books
- Created "Identity Papers Full by Sub-Topic" list
- Created "Identity Read Papers by Sub-Topic" list
- Created "Papers by Title within Topic" list
- Jump-Tables: added "by topic" option
- Bridge
- 25Q1
- Bridge Results Export: Correct residual problems following code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug. To fix, I simply added a check that if the link to the sessions results for the final row isn't blank, I issue a warning.
- 24Q4
- Correct processing for Partner's NGS calculation
- 24Q3
- Bridge Results Export: Make code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug. The symptoms:-
- After run, while the database closes OK, MS Access remains active and had to be crashed via Task Manager.
- If the Export is run a second time, as is required for Partner's calculated NGS to show correctly, without closing the database first, we got 'permision denied' when trying to create Bridge_Results_Theo.htm.
To fix, I only create the hyperlink to the results if the link is blank. This is a bit of a bodge, and I may revisit it later.
- Small Systems: Make code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug.
- 23Q4
- Added session-counts by partner to consolidated results
- 23Q1
- Automate NGS Data Gathering & Recalculation
- 20Q1
- As revealed by Spider: Bridge_Code_Archive.htm (14 items) & Bridge_Code_Archive_Recent.htm (5 items)
- 19Q3
- Add Mountnessing archive links to Results
- NGS Refinements
- Synchronise Documentator with main website generator
- 18Q3
- Reformated + Fixed Hand-dealing pages to CSS standards
- 18Q1
- Corrected format of Links from Bridge Page
- 17Q2
- Created hyperlinks from my consolidated results pages to the actual results
- 13Q2
- Created “Consolidated Bridge Results” webpages (for my own results).
- Clock
- 16Q2
- Created a "chess clock" application to help time tasks. Not much use in the end.
- Data
- 15Q4
- Corrected data content on Identity Sub-topics
- Documentation
- 23Q4
- Documenter
- 20Q2
- 18Q4
- Outstanding Developments by Category appearing all in Bold, and links to lists failing.
- 18Q3
- Created Functor using Form_Documentation_Links to automatically add control-links to documentation Notes.
- Fixed Bug where a variable has the same name as a procedure, and is thereby confused with one. Produced a warning query (Variable_Code_Name_Clashes) for manual correction.
- 18Q2
- Investigated the error reports from the Documenter:
- Improved the diagostics
- Removed unused variables from the Generator code.
- 18Q1
- Created consolidated Development Log, and updated "Progress to Date" and "Outstanding Developments" Reports
- Documenter Bug-fixes: Objects in Comments, MsgBoxs & Debugs
- 17Q4
- Code Archive: Created Web-pages of Recent Code Changes
- Code Archive: Hadn't been working - fixed and recovered a selection from backups
- 16Q1
- Created an automatic Forms documenter – with links to the code.
- 15Q4
- Segregated the code-archive, which was bloating98 the generator database.
- 14Q2
- Created multiple secure areas on my website, so that Documentation is no longer hiding under my Supervision Notes.
- 13Q2
- Created a code & query archive facility as part of the Code Documenter.
- Wrote the Code Documenter.
- Footnotes
- 18Q3
- 10Q2
- Added intra-Note footnotes – a real breakthrough (strange I’d not thought of this before). It will enable me to convert to Note form all the old .pdfs from my undergraduate days – which are replete with footnotes – if this is any use.
- 07Q3
- Improved transparency of Footnote referencing in Printable Notes
- 07Q2
- Created auto-backup of Footnotes101, and more efficient maintenance web-routines
- Created auto-renumbering102 routine for Footnotes
- General
- 19Q2
- Created 'Chess' Home Page
- Created as a main menu item and Note97
- Created consolidated results pages
- Added games & links to pgn4web
- 18Q2
- Website Tidy-up:-
- Wrote a function to delete - with specified exceptions - items not regenerated in the last full website regeneration.
- Tidied up Site - deleted non-updated pages (especially old Printable Notes). Automatically on the local site, manually - using the audit log - on the live site.
- Investigated setting up an FTP link as a drive so I could automatically delete from the live site, but couldn't get it to work. Easy to set up as a drive, but not to assign a drive-letter so the drive can be used by VBA.
- 12Q3
- Improved Titles for Search Results display optimisation
- 07Q2
- Improved "Disclaimer104" Page. Reduce font, add ref to philosophy databases + hyperlinks
- Infrastructure
- 24Q4
- Migrate 2 x Laptop + Desktop from Norton AV to Microsoft Defender: Norton non-renews 13/11/24
- 24Q3
- Migrate from Outlook to New Outlook (3 PCs)
- New Colour Inkjet Printer: Investigation, Purchase & Installation of Epson EcoTank
- 23Q3
- iCloud for Windows: Re-install & solve 'The upload folder for iCloud Photos is missing' problem. Try on new Laptop.
- 23Q2
- Toshiba Laptop - Coffee over keyboard! Fixed - disabled inbuilt keboard + installed USB keyboard.
- 22Q4
- Commissioning Bing with favourite selection of live own-website pages
- Migrate Windows Live Mail POP data on Toshiba to Outlook on Lenovo
- Researching, Purchasing & Installing Palincorp Desktop PC, Keyboard & Screen
- 19Q3
- PDFelement: Purchase & Commission upgrade from Version 5 to Version 7 for $39 - await new Laptop
- Purchase & Commission new Lenovo Laptop: awaiting replacement following motherboard failure!
- 18Q1
- Migration of Website from UKServers to Namesco. Test out.
- Notes
- 24Q4
- Active Note Groups with Notes Updated in the last 7, 14, 28 days: producing incorrect results. Important as there are links from my Home Page. Added Temp_Note_Timestamp column to Notes table.
- 24Q2
- 24Q1
- Enhance comparative Ling database - Output Vocabulary and Dialogue pages for individual languages (so that it's possible to revise them on my Iphone)
- 23Q3
- Investigated & implemented shorter & meaningful URLs for significant Notes
- 23Q2
- Ling database: Reformated webpages for iPhone
- Enhanced comparative Ling database - Added Polish to Ukrainian vs Russian
- 22Q4
- Amended Brief Thoughts on Language & Languages109 to have 4 priorities: Added non-Ling languages
- Reduced size of Aeon Page110 page - which had become difficult to load - by suppressing all the footnoting when a Paper had been generated (and which therefore contains the same information). Retained the link thereto.
- 22Q3
- Installed Espanso - and created its database: Espanso. This enables automatic addition of Note hyperlinks (without having to remember or look up the IDs) when using a text editor.
- 22Q2
- Add Banner Menu for links to Citations, etc, as for Papers
- Add buttons for Lookups for Note IDs and Snippet IDs
- Added %age read to PID Reading Lists for 'Unread' items
- Fixed bug in Reference_Notes checking for C++ false hyperlinks (C++NP string).
- Fixed error in repeat citations in AddCitations_List
- Segregated PID Reading Lists - The Cut
- 22Q1
- Amended Brief Thoughts on Language & Languages111 to have 3 priorities
- Defaulted Year & Month in Website Development Log & Oboe Reports
- Enhanced comparative Ling database - Added Ukrainian vs Russian
- 21Q3
- Create a page to compare the vocabulary across related languages on Ling
- Note Snippets
- Created a method of using the same text in multiple Notes.
- Created Note_Snippets table.
- Created Functor_24, with some control over the bullets.
- Currently only used in Thesis - Chapter 01 (Introduction)112 and General Surveys113.
- May require enhancement as it gets more widely used.
- 21Q2
- Develop a method to create Papers for rows in Aeon_Files that have an associated Identity_Note_ID but no Paper_ID.
- 21Q1
- Developed a mechanism to make the reading lists in my PID Notes easier to maintain and transfer to the Chapters on my Thesis:-
- Created a new table to hold the reading lists (include some static so it's easy to maintain): PID_Note_Reading_Lists
- Created a new Functor to use this table to generate the reading lists within a PID Note (automatically segregating "read" from "unread": allowed to run with two options to allow for comments between the lists: Functor_22, Options 6 and 7, respectively).
- Enabled footnotes for reading-list items.
- Created one-off queries (PID_Notes_Books_Referenced_Gen & PID_Notes_Papers_Referenced_Gen) to extract the reading lists from PID Notes and populate the new table (using the Cross_Reference table)
- To prove the process, started manually to replace the Reading Lists in all the PID Notes with the Functors (completing this will be a manual exercise and, after each run, will involve checking Live versus Test sites).
- Procedurally, in the Thesis Chapters, if Notes are referenced in the Text, use the "NP" encoding so that they don't contaminate the reading lists: only those explicitly listed should do so.
- Added flags to either exclude Books or Papers logged in PID_Note_Reading_Lists altogether, or to segregate them as "missed the cut" in the Thesis Chapters (and, possibly) the Notes themselves.
- Automatically update table PID_Note_Reading_Lists to maintain the "read status" and Book/Paper Title.
- Developed a mechanism to monitor usage of PID Notes within my Thesis (by Chapter):-
- Created Note 1317114, which incorporates as table and a list, created as follows ...
- Used Notes: Functor_21, option 14 (Query PID_Notes_Used_By_Thesis_Chapter_Xtab)
- Unused Notes: Functor_22, option 5 (Query PID_Notes_Unused_By_Thesis)
- Enabled multiple parameters for Functors
- Enhanced Functors to work for selected non-temp Notes (ie. PID Notes: ensured Functor_22, Options 6 and 7 can run on a non-Temp Note, without creating an Archived Note; borrowed code from Auto_Reference_Notes) so that up-to-date stats can be incorporated. This was fairly complex as I wanted to avoid production of an Archived Note each time a non-Temp Note is regenerated. I also wanted to ensure that Notes whose variable text consists entirely of Functors get archived correctly (which they might not have if I'd removed Functor-generated text from Notes before saving them to the Notes table; in fact, the text saved to Live and Archive is identical and includes the full Functor content).
- Enhanced the mechanism to make the reading lists in my PID Notes easier to maintain:-
- Made the table PID_Note_Reading_Lists easier to maintain: added a menu at the start of the Create Notes Web Pages115 to (inter alia) open the PID_Note_Reading_Lists table, add or delete rows, change the "excluded" and "cut" flags, and amend Footnotes. Book/Paper titles and read-status are automatically maintained by the Functors.
- Added functionality to allow the reading lists to be further segregated - as in the Notes Animalists116 and Christian materialism117 - which din't use the Functors (though their reading lists are on PID_Note_Reading_Lists).
- Created a further Functor (Functor_23, Option 17) to generate the Thesis Chapter Reading Lists from the Notes explicitly listed as relevant in the new table Thesis_Note_XRef. Uses Functor_Indented_List, which also creates (via Functor_23, Option 16) the indented PID-Note lists. A major triumph!
- Modified Functor_21 Option 3 to allow multiple Audio Files per timesheet line
- 20Q4
- 20Q3
- Added Aeon Abstract (ie. Footnote) link to Aeon Webref items in Summary task List reports for 'read' items
- Added option in Auto_Reference_Notes to - for Live (ie. not TEMP) Notes - update the latest Archived Note to equal the updated Live Note, hence avoiding superfluous archives.
- Added option in Auto_Reference_Notes to only confirm new items (leaving previously-flagged items untouched)
- Corrected Issues with Nested Functors: Added Functor_ID to End-tag
- Determined why the run of Auto_Reference_Notes_Regen took 2.5 hours on 02/09/2020 (and climbing). It was down to the Aeon Note121 being searched; in particular Check_If_In_Container, checking for items in Footnotes and / or Quoted Text. As this Note is regenerated, there's no point auto-linking it. Time reduced to 23 minutes once this Note - and other Notes of inappropriate Note Groups - was removed.
- Highlighted Archived Notes clearly as not the latest
- Upgraded Auto_Reference_Notes to allow a parameterised 'termination or continue' for very long update runs
- Upgraded Auto_Reference_Notes to avoid key-words within +AA+-style Authors names (using Check_If_In_Container).
- Upgraded Auto_Reference_Notes to log its actions to a new table (Auto_Reference_Notes_Actions) so that any errors can be investigated and improvements made.
- 20Q2
- As revealed by Spider: Sundry uncategorised. Refs failing. 30 items. Fixed manually.
- 20Q1
- As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
- As revealed by Spider: Blackberry links from archived Note 1010 (Maintenance Dashboard). 38 items.
- As revealed by Spider: Blog122 Achived Notes - sundry links to Printable Notes. Live Note OK. 97 items.
- As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
- Solve the problem of the size limitation for Notes by allowing their concatenation.
- 19Q4
- Create procedures (manual and via Functor_17) to add Audio Files to Notes
- 18Q4
- As revealed by Spider: Consciousness Status Report123 Achived Notes - links to BookCatalogCategorised_16_115_471.htm failing. Live Note OK. 46 items. 24 items outstanding.
- As revealed by Spider: Links from Blog in Level 1 or 2 Printable Notes 512124 and 981125 failing. 141 items.
- As revealed by Spider: Links to sundry 'static' pages failing.
- 13 items fixed.
- 7 items outstanding: to do with Pete's PhD Test site. Links seem to be OK, so may be a problem with the Spider. Directory "Petes_PhD" was the only one set to "do not parse". The file wasn't in the site map. Reset; hopefully it'll work!
- As revealed by Spider: Status Reports - Achived Notes - sundry links failing. 50 items in total. Live Notes fixed.
- Website: Maintenance Dashboard126. 14 items.
- Personal Identity Status Report127. 11 items.
- Summary Status Report128. 4 items.
- Actual Detail Summary Report129. 6 items.
- Chess Status Report130. 12 items.
- Etc. 3 items.
- As revealed by Spider: Sundry links failed for Supervisions. 30 items.
- Create Priority Task List Report131.
- 18Q3
- As revealed by Spider: Bridge Status Report132 Achived Notes - sundry links failing. Live Note OK. 70 items.
- As revealed by Spider: Languages Status Report133 Achived Notes - links to Dissertation failing. Live Note OK. 43 items.
- As revealed by Spider: Religion Status Report134 Achived Notes - sundry links failing. Live Note OK. 59 items.
- As revealed by Spider: Links to Notes_xxx_Links, from Abstracts (94 items), Notes (25 … corrected manually), NotesPrint (1916). Fixed by +LL+ functor implementation.
- As revealed by Spider: Notes_List_Control links fail for Supervisions. 58 items.
- Automatic Key-word Hyperlinking to Notes Project: Phase 2
- Added links based on Book & Paper Titles
- Added ***Read***-marker to Lists
- Allowed links to be ignored (rather than suppressed) during approval process
- Skip links to Status Reports, etc.
- Blog: Created Functor to generate Blog from Blog.
- Created auto-regen of all Note Link (Note_nnn_Link) Pages
- Created new '+LL+' Functor for links to Note_Link pages; fixed associated Notes.
- 18Q2
- Added a process to 'fix' - by adding the Note-ID - those note-links added without stating the ID. This will save processing time.
- Created automatic key-word referencing (ie. to Notes in the Notes-Group "Personal Identity"). This was complicated for a number of reasons:-
- There are always false positives, so there needed to be a way of "proofing" what's to be done before doing anything.
- There will be times where it is inappropriate (eg. in quoted work by other authors). So, there had to be a way of permanently suppressing such links.
- The referencing differs depending on whether the term is in a Footnote or not.
- I needed to check for already-existing links, variant grammatical forms, etc.
- I created a MsgBox routine - providing context - so that what needed to be done with potential links could easily be decided when the linking process was (re-)run.
- Links are always to Notes, but are from Notes, Archived Notes, Authors and Book & Paper Abstracts & Comments.
- 18Q1
- 17Q4
- Added %age complete to Notes Reading Lists
- Enabled deduction of Note link by omitting the Note ID from the mark-up and doing a look up on the key-word and its varients.
- 17Q3
- Added Colour-coded "quality markers" to Notes: included them in lists and jump-tables
- 16Q1
- Allowed Note- & Footnote-links from multiple words144. This is such a useful feature!
- Completed converting “static” pages to Notes.
- Enabled direct links to Notes from within Footnotes.
- Stopped (I hope!) “private” Notes showing in Paper Abstracts.
- 15Q4
- Converted the “static” pages in my website to Note format (so they can change and retain history).
- Fixed problem of images not appearing in Printable Notes (wrong directory level) unless the images are repeated
- 15Q3
- Tidied up the archived Notes.
- 14Q4
- Made improvements to the regeneration and referencing of archived Notes.
- 14Q3
- Created a progress-tracker142 for my Thesis.
- Improved error-recovery where encoding fails for Notes links.
- Started to split Paper Abstracts from Write-ups.
- 14Q2
- Wrote routines to import old (but missing) archived Notes into the database so they can be (and have been) re-exported in the current format.
- 14Q1
- Made a lot of improvements to my Site generators, in particular the pesky bug that stopped the links to archived Notes working correctly.
- 13Q3
- "Single Image" development for Notes
- Allowed "Authors Works Lists" to be linked-to from Notes
- Created Archived Notes Re-Export facility
- Simplified / Re-engineered Notes processing
- 13Q2
- Improved "Concatenated Notes" production
- 10Q3
- Add “Date Created” to Note_Links (for audit purposes; I’m still not convinced that links disappear when they should).
- Investigate Note_Links: needs versioning, and section of calling Note
- Wrote routines to automatically generate the “Printable Notes” footer – and to create the corresponding web pages – from the control table. Very useful.
- 09Q4
- Restricted the size of the “previous versions” tables for Notes to 12.
- 09Q3
- Enabled automatic immediate promotion to Live Website of selected Temp Notes.
- Enabled Concatenated Notes (by Group). This is only allowed for “Identity” Notes at the moment, which is what I’d wanted it for. It allows all Notes for a Group to be printed off in Title sequence, for ease of review.
- Ensured that the Notes Reading-list is generated from all 3 sub-topics.
- Functionally enhanced the Notes-generation processing options; to make niftier production possible. In particular, where individual Notes are selected, I made the re-generation of the cross-references optional (so that if only textual variation is being iterated, the iterations are quicker).
- Improved the performance of “All Notes” regeneration. The problem was with the generation of the generic reading lists based on Note Title. These only apply to the “Identity” Notes, and it was taking a long time to determine there was no list to produce.
- 09Q2
- Added a table of Notes history at the end of each Note (ie. not just previous version) - to save tedious chaining
- 09Q1
- Added reading-list (of works cited) at the end of the Note, as well as in the Printable Note
- Created optional "don't print" feature for specific footnotes.
- Fixed the problem of the overpopulation of the Notes directory
- Option for Notes-Regen:
- Changed-only
- Changed, and all impacted by Change
- All
- 08Q1
- Allowed regeneration of Temp Notes only
- 07Q4
- Added referenced Books / Papers as footnote jump table to Notes
- Added referenced reading lists to printable version of Notes
- Improve "Temp" note processing. Suppress "_T" suffix (just don't copy to Transfer)
- When a Note is archived, ensured a printable note is also archived146 (if this Note has a print)
- 07Q3
- Created "respondent notes" - database identifier & red indicator + text
- Created147 a "quick lookup" table of responses to a note and its sub-notes (recursive function on Notes_Links)
- Enabled generation of printable version of Notes
- Made Note-change-history available on-line (added status + publishable flags)
- 07Q2
- Added "footnote" pages148 for Carthusians & King's
- Added "outgoing links" summary footer to Notes pages
- Allowed two sorts of Note - document-specific ("Write-up") and general
- Papers
- 21Q2
- 20Q3
- 20Q2
- Determined why the monthly regeneration process for Paper Abstracts had ballooned from just over 6 hours in January 2020 to over 17 hours in April 2020. Reduced to just under 4 hours as a result of materialising and indexing the view (now table) Paper_Citings_List_New. Further improvements required.
- Determined why the monthly regeneration process for Paper Summaries had ballooned from just over 3.5 hours in January 2020 to around 11.5 hours in April 2020. Fixed by materialising and indexing the view (now table) Paper_Citings_List_New, so now takes 6 minutes!
- 20Q1
- Make use of Comments for large Abstracts seamless by appending Comments (if > 5k) to Abstracts.
- 18Q2
- Papers associated with broken links:-
- Created an area to hold pdfs, and automated links thereto.
- Created a pop-up to say I'd remove such pdfs if requested by their authors.
- Consequently, haven't password protected the area yet.
- 18Q1
- Correction of narratives on Note time-recording pseudo-papers
- Photos
- 19Q1
- Created Photo-logging usage via Images table.
- Created Timeline software
- Timeline software: Added photos of Coxes Farm
- 18Q3
- 13Q2
- Reduced the size of photos (to make the pages quicker to load).
- Process
- 20Q3
- Determined why Recalculation (cmdRecalculate_Click) took so long: it varied depending on how many Notes were regenerated, but seemed to be 17.5 minutes even if there are none. I think it was down to a problem with nested Functors leading to Notes growing inordinately: in particular the development logs. Now takes around 5 minutes.
- 20Q2
- Full Website Regeneration took 60 hours in April 2020. This has now been substantially fixed, as of end June 2020, in that regeneration now takes just under 11 hours. While further improvements are possible, the best use of time is to avoid the need for regular regeneration altogether by completing my Cross-referencing project.
- Corrective work was split out into the items requiring improvement, must of which have now been implemented.
- History and analysis:-
- This 'balooning' was on my new laptop which has solid state disks - it was taking 36 hours on my old laptop, then halved when I got the replacement, but then nearly doubled!
- I investigated why this was so, and improved performance mainly by putting in a trace (using a timer - GetTickCount - found on the web that allows logging of elapsed time in milliseconds, using StartTimer and EndTimer) and finding the 'pinch points'
- I suspected that it was a new release of MS Access that's caused a problem; maybe a lost index or something similar. I couldn't find any evidence of this, but a mumber of queries were taking a second or so to execute, which small times multiplied up drastically when the number of pages on my site - over 100k - are taken into account. Quite why a cliff-edge had been reached, I know not!
- Three areas had been particularly impacted: Authors, Paper Abstracts & Paper Summaries, which have all trippled or worse. These are covered as seperate developments under 'Authors' and 'Papers'.
- Book Summaries and Book-Paper Abstracts had also at least doubled, but this has had less of an impact as they took much less time in the first place.
- Other regeneration processes - in particular 'Notes' - didn't appear to have been impacted.
- Wrote Check_Database_Size to check that the size of the database isn't at risk of breaching the 2GB limit. Required because of the need to re-materialise various views that have been instituted to speed up the full website regeneration routines, and which need to be run each time cmdRecalculate_Click is run, which is often.
- 19Q4
- Fix Functors to enable previous quarter's Status Reports after Time period roll-over
- 18Q4
- 18Q2
- Re-engineered Website Maintenance Dashboard149 to show extra topical links (and show them at the top of the page).
- 18Q1
- Improved efficiency of daily changes regeneration (removed msgboxes)
- Wrote a routine to update a set of .htm files, replacing text in them all
- 15Q4
- Created a consolidated processor to enable overnight regeneration of the whole website.
- 14Q1
- Created a “Website Regeneration Dashboard” to show when the various elements of my website have been refreshed, and how long the routines took to run. Links to this appear in the footer to each webpage.
- Improved the regeneration routines for the various classes of webpage, and got them to record statistics in support of the “Website Regeneration Dashboard”.
- 09Q4
- Set up my database system to roll forward to the next academic year, and generate web-pages appropriately.
- 09Q3
- Created a “performance stats” database and routines to write to it – though I only implemented them for the Notes generator.
- 07Q2
- Improved ease of FTP process by use of Transfer Directory
- Search
- 17Q2
- Wrote routines to create a cut-down Website excluding pages with irrelevant or duplicated information. Used this to create the ZoomSearch facility.
- 07Q4
- Created search facility using ZoomSearch
- Spider
- 22Q4
- As revealed by Spider: "Aeon - Video - Organism" had author set to 'Metamorphosis' (with Note link). Maybe a manual error, combined with Espanso? Fixed 15 impacted pages Manually.
- 21Q3
- Improved Spider performance by removing all Printable Notes - this will need to be reversed sometime.
- 20Q4
- System Resources Exceeded - "Run Time Error 3035" - re-emerged in Full_Link_Same_Directory_Gen. Last month I had replaced query Full_Link_Same_Directory_Updt with identical SQL, which had worked OK. This time I removed the "WHERE" statements and put them in the code itself. Seems fine.
- 20Q3
- System Resources Exceeded - "Run Time Error 3035"; query Full_Link_Same_Directory_Updt in Full_Link_Same_Directory_Gen. Fixed by using SQL rather than a query.
- The size of the main database was bloating to over 1.5Gb during the spider run, so was approaching the 2Gb limit.
- Used Check_Database_Size, with a parameter, to monitor the size of the main database and added a message similar to those reporting the compact / repair of the Slave database each time a status message is printed.
- Based on the diagnostics produced, it was found that the bloating occured at two stages:-
→ In Spider_Scurry: during the creation of Raw_Links_Temp_Temp.
→ In Spider_Copy: during the creation of Raw_Links_Temp.
- Fixes:-
→ Put check in Check_Database_Size to STOP if the database is over 1.5 Gb (parameterised via Max_Database_Size).
→ Moved Raw_Links_Temp_Temp to the Slave database.
- 18Q4
- Determined why Full_Link_Up_Levels_Gen takes so long - approximately 4 hours. Re-architected to reduce database reads.
- 16Q4
- Added “.shtml” and “.shtm” to the list of web-suffices.
- 15Q4
- Finally cracked the Spider for external links, and used it to check the 3,000-odd links from my website. Some tidying up outstanding.
- 14Q4
- Fixed the bug wherein the Spider ran out of system resources on large runs: now compacts & repairs the slave database as it does along (every 200k links; parameterised).
- 14Q3
- Continued working on my Spider and started to use it to clean up my website by diagnosing broken links.
- 13Q2
- 10Q2
- Wrote my own Web Spider.
- It’s a little bit of a cheat, in that it doesn’t actually invoke the web-links, but crawls through my directory structure selecting all the .htm files, and rummages through the HTML looking for links which it then copies to a table.
- It also maintains a table of the directory structure and files therein, and then works out the real address from the relative addressing.
- It then allows cross-checking between the links and the files.
- Links external to the site have to be investigated manually (at least I have a list of all the links) – unless I can work out how to use the list to power a browser to check.
- There are about 333k links in 55k files – the full run took about 8 hours.
- It now runs incrementally based on file last-updated dates.
- Status
- 24Q2
- 24Q1
- 23Q1
- Add a Functor to track how my daily time distribution tracks what I plan to do. Add a table of hours worked / weekday (QTD & YTD / Actual versus Plan) to Status: Summary - Actual versus Plan154.
- 22Q4
- Bible Reading Status Reporting: Add Koran
- Fixed Year-End changes required for Oboe 'practice since last lesson'
- 21Q3
- 20Q4
- 20Q3
- Added Weekly Project Plans to Priority Task List
- Created web-page (using Functor_21) showing oboe practice hours by work played
- Quarterly Project Reports: Corrected Functor_08. The Project Planned YTD % kept having to be bodged!
- 19Q2
- Corrected Quarterly Reporting for uneven Plan (Holidays)
- 18Q4
- 18Q3
- 18Q2
- Added extra items to Functors to facilitate Quarterly Reporting:-
- Automatically calculated QTD & YTD time statistics in the quarterly reports.
- Provided a YTD task list for Status Reports for projects "in abeyance" to make reporting easier.
- Modified the statistics - to be just YTD - for YTD Status reports.
- Created table for - by Project - "Plans for the near future" tasks, and a functor to add them to the status report.
- Took "items competed" and "plans for the near future from the development log for the Web-tools status report.
- Created another functor to add these lists to the Summary report.
- 18Q1
- Created web-page generator for Development items (ie. Created the Web-Project "To Do" List and "Outstanding Developments" directly from this list).
- 13Q3
- Re-engineered Monthly Reporting on Change of Year
- 13Q2
- Improved "This Month's Summary" Reporting
- Improved Quarterly Reporting
- 13Q1
- Re-engineered Monthly Reporting Tables & improved Monthly Reporting
- Technology
- 20Q4
- Consolidated Site-Hits:
- Reviewed Webalizer Methodology & Consolidated Stats
- Aggregated Robot URLs
- 19Q1
- Investigated the Hit Stats provided by Webalizer and produced web-pages to show the most active pages.
- 13Q3
- Converted Tables to CSS3 standard
- Used CSS in all manually-crafted and automatically-generated pages
- 13Q1
- Timesheets
- 17Q4
- Separated Time Recording for Aeon
- 15Q1
- Segregated time-recording so I can easily see how much time is spent developing and supporting my own website, and how much is spent on those of other organisations.
- 13Q3
- Re-engineered Time-Recording, so that actuals are held against Papers (and Books) in a separate table, rather than as columns by year in the Papers / Books tables themselves. Retained the grand totals only.
- 08Q4
- Updated Book totals from totals of the Papers contained therein
- WebRefs
- 22Q4
- Hits_Pages_Total_Crosstab_Descending.htm & Hits_Bots_Total_Crosstab_Descending.htm: Pages too large to load. Fixed by:-
- Restricting page rows to total hits > 99 (this was intended, but had got lost somewhere).
- Summarising by year (rather than by month) for years before the current one.
- 22Q2
- Checked whether the WebRefs checker still works now IE has been retired. It does, but I may have a problem one day when I need to install IE on a new PC.
- Fixed Bug in Webrefs_Update where dud Wikipedia pages not recognised
- 21Q1
- Webrefs_Update failed because IE looped with Aeon / Psyche pages. Performed manual checking for a while, but then Aeon seem to have fixed their pages so the problem seems to have "gone away". It doesn't occur for other pages.
- 20Q3
- Improved WebRefs checker (Webrefs_Update) further to check for Error 403 "Forbidden". This involved finding way of checking pdfs where the returned page is in fact HTML or XML (see DevLog Ref 379).
- Reformated WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: The 'As Above" lines waste space. Consolidate onto single second line.
- Reformated WebLinks_Tester_Brief, WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: Allowed more space for 'link returned', 'issue' and 'display text' and added Explanation column
- 20Q2
- Amend WebRef 'Name' link to #Off-Page_Link_WxxxW style. Also amend Quarterly and Annual task lists to use this reference.
- Amended WebRef 'Name' links to #Off-Page_Link_WxxxW style so referencing from other pages is possible (useful for Aeon)
- As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 184 items. res://ieframe.dll/ in Returned_URL.
- WebLinks_Tester: If the URLs of WebRefs are changed between Spider runs, the URL mapping fails, and the WebRefs appear at the end of the last page, wrongly categorised. Fixed to ensure they appear on their own page, added to the jump table as 'WebRef Missing' and added an explanation.
- WebLinks_Tester_Full & WebLinks_Tester_Full_Map: Reformated Jump Table to 30 columns and multiple rows.
- 20Q1
- As revealed by Spider: WebLinks_Tester_4.htm. Refs failing. 32 items. res://ieframe.dll/ in Returned_URL.
- As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 51 items outstanding. res://ieframe.dll/ in Returned_URL.
- Look into Sistrix Smart166. Errors and warnings itemised are:-
- Duplicate content: seems to be variants on theotodman.com
- Title Tags: Empty, too long, identical
- Page Not Found
- Filesize in excess of 1Mb
- Meta-Description: Empty
- Few words on Page
- H1: Not used, used multiple times per page, identical across pages
- Pictures: Alt attribute missing
Since at least August 2022, Sistrix has only produced 3 errors: home page not accessible, therefore no other pages accessible and also not HTTPS.
- WebRefs checker failing to detect broken links for Aeon - if a link fails, I get the message 'The page you were looking for could not be found (404)', but since the URL doesn't change, I don't spot it. This may happen for other sites as well. Fixed for Aeon - watch out for others.
- 19Q3
- As revealed by Spider: Hits_Pages_Total_Crosstab_Descending.htm.
- 18Q4
- Added Display text to WebRefs for Links to Stanford, Wikipedia & Aeon
- 18Q2
- 18Q1
- Determined & reported usage directly using embedded +WnnnW+ encoding rather than via the Spider. Reconciled the two methods.
- Documentation & Bug-fixes: Phase 1
- Completed "as is" functional documentation so I could see what's going on.
- Re-engineered so it made sense and ran efficiently without bugs
- Fixed the obvious data errors that confused the statistics - ie. where the returned URL differs from that requested and https issues.
- Wrote / improved diagnostic reports (web pages)
- 16Q1
- Some tidying up of the 3,000-odd external links from my website.
- 13Q2
- Some pages render oddly in Chrome (and maybe other browsers) – investigated168.
- 10Q3
- Improved the functionality for automatically generating external Web-links in Notes, Book Abstracts and Paper Abstracts. Corrected current data to accommodate the new code.
- 07Q2
- Made Web Refs in Comments / Abstracts into Hyperlinks
- Other Websites: Completed Items By Category:-
- A1Wasps
- 25Q1
- a1waspsandbees: Invent method of getting the site higher up the Google search list without paying!
- 24Q4
- Acts
- 16Q3
- Developed a website for displaying the textual and grammatical analyses and appendices of Pete’s PhD on the Acts of the Apostles. Two versions: Live (Acts: Live Site (Defunct)) and Test (Acts: Test Site).
- Bridge
- 25Q1
- Old House Bridge Club Website: Set up for 2025 and permanently resolve the 'too many databases' problem.
- Small Systems: Make code changes to work around the ‘Cannot open any more databases: Run-time error 3048’ bug. I improved the fudge that simply re-creates Small Systems for the current year by removing the hard coded '2024' and replaced it with 'Current Year'. Should be adequate. Seems to work fine.
- 23Q3
- Read and thoroughly understand the documentation on:-
→ EBUScore
→ Bridgemate II
- 23Q2
- Revived Small Bridge Sites
- 21Q3
- Major Spring-clean of the Mountnessing Bridgewebs site.
- 17Q1
- Added new features (70s Club & Monthly Pairs Winners) to my Bridge mini-websites.
- Created Master Point Lists & Promotions for the ECBA website.
- 16Q3
- 16Q1
- Accommodated Alaric’s new scoring system for St. Edith’s.
- Created Ladies’ and Gentlemen’s ladders for my mini-websites.
- Reconciled my X-IMP calculations with a new feature provided by Bridgewebs.
- 15Q4
- Segregated the Bridge website generators from those for my own website. They now live in separate databases and have their own documenters.
- 15Q3
- Created a new website (using Bridgewebs) for the First Class Bridge Academy (First Class Bridge).
- Ported my “handicap / improvers” routines from Hutton to “Mini Websites”.
- 15Q2
- Development of the “Eve Goblets” (victor ludorum) software for the ECBA, and roll-over to the new season.
- 15Q1
- Further improvements to my Butler-IMPs and Cross-IMPs calculation routines.
- Started a project to provide small websites for bridge clubs without sites: Essex Bridge Results.
- Took on the ECBA Website (ECBA Website (Defunct)), at the cost of considerable effort by way of reformatting & tidy-up.
- 14Q4
- Improvements to my Butler-IMPs and Cross-IMPs calculation routines – different formats, including the Jeff Smith format, and extension to Butler-scored Pairs events. This work has been taken up by the ECBA selectors for county-team selection.
- 14Q3
- Created a suite of programs to extract data and calculate Butler-IMPs and Cross-IMPs from the travellers on Bridgewebs Teams events, and output ranking-lists to the Web.
- 13Q2
- Merged169 Philosophy & Bridge Databases
- Enigma
- 25Q1
- 18Q4
- Added 'EE' shortcut icon on all Webpage browser tabs
- 15Q3
- Created a small website for a music group Julie and I attend – the Enigma Ensemble.
- Sophie
- 19Q3
- 18Q4
- Added 'SB' shortcut icon on all Webpage browser tabs
- 15Q3
In-Page Footnotes:
Footnote 6:
- There have been continual changes and bug-fixes that are not worth reporting.
- A quarterly release - or the equivalent - of small changes is to be understood passim.
- There were gaps in the development log that have been filled in from old reports. I've ignored developments for now-defunct sites.
- However, for my own site I've erred in the direction of plenitude to remind me of what went on. I may prune the list in due course.
Footnotes 41, 166:
- See Sistrix
- This used to be called Optimizr, see Optimizr (Defunct) (which now auto-forwards to Sistrix).
- A quick look doesn’t show it to be an obvious scam, but I need to double-check.
- An unsolicited analysis of my site turned up monthly from Optimizr from January 2015 to October 2017, listing a large number of “problems” that I think I know about, but which are in the queue to address.
- It restarted in February 2018, under the Sistrix name (this seems to have been associated with Optimizr since November 2015).
- The free version of this software is restricted to 1,000 pages, which is a very small proportion of my Site, though I may be able to point it to difference base-URLs.
- But I do need to address the problems validly itemised, and a sub-set is still useful.
Footnotes 56, 99:
- This situation occurs in these reports, and elsewhere.
Footnotes 57, 100: Footnotes 66, 137:
- An eccentric use of this term.
- By it I mean the extraction of data from a database, formatting it, and inserting the resulting text in line into a web-page.
Footnotes 73, 144: Footnotes 74, 98:
- I subsequently found a bug whereby it was backing up the entire code-base each time!
Footnotes 75, 170:
- Sophie is an old friend and supervisor from Birkbeck.
Footnotes 77, 91:
- Naturally, I can now say precisely what I mean, by using a plus-sign, as it no longer causes the process to fail!
Footnotes 78, 168:
- Fixed – it was very long blank-free URLs in the Source column.
- I now call my “WebRef” function to convert these to hyperlinks.
Footnotes 79, 169:
- This was a mistaken idea, and was subsequently reversed!
Footnotes 80, 165:
- Rather irritatingly, when I checked my site in all browsers, I found that it appeared a right mess in some of them (I forget the details).
- It all stemmed from the DocType parameter, which I had to revert.
Footnotes 81, 146:
- I must have unwound this one - presumably it didn't work - and need to re-do it.
Footnotes 82, 147:
- Did I - or was this an intention?
- The whole "respondent" process didn't work out well as it fragments the discussion, so this might be important.
Footnotes 83, 92:
- This must have been unwound if it was ever implemented.
Footnotes 84, 94:
- "BB" = "Blackberry" (for when I had one!)
- Response times - eg. in bookshops - used to be awful. This "binary chop" approach is no longer needed.
Footnotes 85, 86, 88, 101, 102, 148: Footnotes 87, 104:
- Presumably this was the ancester of this page before it was converted to a Note?
Table of the Previous 12 Versions of this Note: (of 30)
Summary of Notes Referenced by This Note
To access information, click on one of the links in the table above.
Summary of Notes Citing This Note
To access information, click on one of the links in the table above.
Text Colour Conventions
- Blue: Text by me; © Theo Todman, 2025