Theo Todman's Web Page - Notes Pages
Status Reports
Website - Progress to Date (2021 - June)
(Text as at 02/07/2021 20:32:38)
*** THIS IS NOT THE LATEST VERSION OF THIS NOTE ***
(For the live version and other versions of this Note, see the tables at the end)
Rationale for this Document
Items Completed In6:-
- 21Q2
- Architecture:
- Created Check_Delimiters to protect against unpaired delimiters in lists, which squiff the formatting of documents that contain them. Output diagnostics to MsgBox, if not a re-gen, and to debug window to enable correction.
- Papers:
- 21Q1
- Notes:
- Modified Functor_21 Option 3 to allow multiple Audio Files per timesheet line
- Enabled multiple parameters for Functors
- Enhanced the mechanism to make the reading lists in my PID Notes easier to maintain:-
- Made the table PID_Note_Reading_Lists easier to maintain: added a menu at the start of the Create Notes Web Pages7 to (inter alia) open the PID_Note_Reading_Lists table, add or delete rows, change the "excluded" and "cut" flags, and amend Footnotes. Book/Paper titles and read-status are automatically maintained by the Functors.
- Added functionality to allow the reading lists to be further segregated - as in the Notes Animalists8 and Christian materialism9 - which din't use the Functors (though their reading lists are on PID_Note_Reading_Lists).
- Created a further Functor (Functor_23, Option 17) to generate the Thesis Chapter Reading Lists from the Notes explicitly listed as relevant in the new table Thesis_Note_XRef. Uses Functor_Indented_List, which also creates (via Functor_23, Option 16) the indented PID-Note lists. A major triumph!
- Developed a mechanism to make the reading lists in my PID Notes easier to maintain and transfer to the Chapters on my Thesis:-
- Created a new table to hold the reading lists (include some static so it's easy to maintain): PID_Note_Reading_Lists
- Created a new Functor to use this table to generate the reading lists within a PID Note (automatically segregating "read" from "unread": allowed to run with two options to allow for comments between the lists: Functor_22, Options 6 and 7, respectively).
- Enabled footnotes for reading-list items.
- Created one-off queries (PID_Notes_Books_Referenced_Gen & PID_Notes_Papers_Referenced_Gen) to extract the reading lists from PID Notes and populate the new table (using the Cross_Reference table)
- To prove the process, started manually to replace the Reading Lists in all the PID Notes with the Functors (completing this will be a manual exercise and, after each run, will involve checking Live versus Test sites).
- Procedurally, in the Thesis Chapters, if Notes are referenced in the Text, use the "NP" encoding so that they don't contaminate the reading lists: only those explicitly listed should do so.
- Added flags to either exclude Books or Papers logged in PID_Note_Reading_Lists altogether, or to segregate them as "missed the cut" in the Thesis Chapters (and, possibly) the Notes themselves.
- Automatically update table PID_Note_Reading_Lists to maintain the "read status" and Book/Paper Title.
- Developed a mechanism to monitor usage of PID Notes within my Thesis (by Chapter):-
- Created Note 131710, which incorporates as table and a list, created as follows ...
- Used Notes: Functor_21, option 14 (Query PID_Notes_Used_By_Thesis_Chapter_Xtab)
- Unused Notes: Functor_22, option 5 (Query PID_Notes_Unused_By_Thesis)
- Enhanced Functors to work for selected non-temp Notes (ie. PID Notes: ensured Functor_22, Options 6 and 7 can run on a non-Temp Note, without creating an Archived Note; borrowed code from Auto_Reference_Notes) so that up-to-date stats can be incorporated. This was fairly complex as I wanted to avoid production of an Archived Note each time a non-Temp Note is regenerated. I also wanted to ensure that Notes whose variable text consists entirely of Functors get archived correctly (which they might not have if I'd removed Functor-generated text from Notes before saving them to the Notes table; in fact, the text saved to Live and Archive is identical and includes the full Functor content).
- WebRefs:
- Webrefs_Update failed because IE looped with Aeon / Psyche pages. Performed manual checking for a while, but then Aeon seem to have fixed their pages so the problem seems to have "gone away". It doesn't occur for other pages.
- 20Q4
- Books/Papers:
- Notes:
- Spider:
- System Resources Exceeded - "Run Time Error 3035" - re-emerged in Full_Link_Same_Directory_Gen. Last month I had replaced query Full_Link_Same_Directory_Updt with identical SQL, which had worked OK. This time I removed the "WHERE" statements and put them in the code itself. Seems fine.
- Status:
- Technology:
- Consolidated Site-Hits:
- Reviewed Webalizer Methodology & Consolidated Stats
- Aggregated Robot URLs
- 20Q3
- Architecture:
- Books:
- BookCitings references not sequenced correctly. Corrected.
- BooksToNotes Link Pages: Re-engineered - used CreatePapersToNotesWebPages to generate pages
- Books/Papers:
- Notes:
- Added Aeon Abstract (ie. Footnote) link to Aeon Webref items in Summary task List reports for 'read' items
- Added option in Auto_Reference_Notes to only confirm new items (leaving previously-flagged items untouched)
- Upgraded Auto_Reference_Notes to allow a parameterised 'termination or continue' for very long update runs
- Upgraded Auto_Reference_Notes to avoid key-words within +AA+-style Authors names (using Check_If_In_Container).
- Highlighted Archived Notes clearly as not the latest
- Added option in Auto_Reference_Notes to - for Live (ie. not TEMP) Notes - update the latest Archived Note to equal the updated Live Note, hence avoiding superfluous archives.
- Upgraded Auto_Reference_Notes to log its actions to a new table (Auto_Reference_Notes_Actions) so that any errors can be investigated and improvements made.
- Corrected Issues with Nested Functors: Added Functor_ID to End-tag
- Determined why the run of Auto_Reference_Notes_Regen took 2.5 hours on 02/09/2020 (and climbing). It was down to the Aeon Note18 being searched; in particular Check_If_In_Container, checking for items in Footnotes and / or Quoted Text. As this Note is regenerated, there's no point auto-linking it. Time reduced to 23 minutes once this Note - and other Notes of inappropriate Note Groups - was removed.
- Papers:
- Process:
- Determined why Recalculation (cmdRecalculate_Click) took so long: it varied depending on how many Notes were regenerated, but seemed to be 17.5 minutes even if there are none. I think it was down to a problem with nested Functors leading to Notes growing inordinately: in particular the development logs. Now takes around 5 minutes.
- Spider:
- The size of the main database was bloating to over 1.5Gb during the spider run, so was approaching the 2Gb limit.
- Used Check_Database_Size, with a parameter, to monitor the size of the main database and added a message similar to those reporting the compact / repair of the Slave database each time a status message is printed.
- Based on the diagnostics produced, it was found that the bloating occured at two stages:-
→ In Spider_Scurry: during the creation of Raw_Links_Temp_Temp.
→ In Spider_Copy: during the creation of Raw_Links_Temp.
- Fixes:-
→ Put check in Check_Database_Size to STOP if the database is over 1.5 Gb (parameterised via Max_Database_Size).
→ Moved Raw_Links_Temp_Temp to the Slave database.
- System Resources Exceeded - "Run Time Error 3035"; query Full_Link_Same_Directory_Updt in Full_Link_Same_Directory_Gen. Fixed by using SQL rather than a query.
- Status:
- Added Weekly Project Plans to Priority Task List
- Quarterly Project Reports: Corrected Functor_08. The Project Planned YTD % kept having to be bodged!
- Created web-page (using Functor_21) showing oboe practice hours by work played
- WebRefs:
- Reformated WebLinks_Tester_Brief, WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: Allowed more space for 'link returned', 'issue' and 'display text' and added Explanation column
- Reformated WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: The 'As Above" lines waste space. Consolidate onto single second line.
- Improved WebRefs checker (Webrefs_Update) further to check for Error 403 "Forbidden". This involved finding way of checking pdfs where the returned page is in fact HTML or XML (see DevLog Ref 379).
- 20Q2
- Architecture:
- Enhanced the Functor processing to allow parameters. Previously, I had to create a new Functor ID each time for a very similar requirement. The main problem with this old method was that these Functors (where they call another Functor, or subroutine from a row in Functor) didn't feed through into the Documenter subroutines properly.
- Truncated Cross_Reference_Changes table (from over 1M rows to under 50k) which should be self-maintaining, but isn't. This seems to have improved the regeneration times of Notes, Archived Notes and some other objects now that insertions into this table are quicker. Needs a permanent solution as part of my Cross-Referencing project.
- Authors:
- Determined why the monthly regeneration process for Authors has ballooned from under 4 hours in January 2020 to over 17 hours in April 2020. Fix on 11/04/20 - time more than halved to 8.25 hours - by re-engineering the 'menu' process that counts the items linked to further down the page.
- Enabled "alternate names" for identically-named authors (to avoid middle-initial = X):-
- Set up and populated Author_Name_Display
- Ensured Author_Name_Display appears on the Author page.
- Further reduced the monthly regeneration process time for Authors from 8.25 hours to 13 minutes! Fixed by materialising and indexing a 'view' to table Authors_Cited_By_All_List.
- Books:
- Determined why the monthly regeneration process for Book Summaries had ballooned from 1.1 hours in January 2020 to 3.5 hours in April 2020. Partly fixed by by materialising and indexing the view (now table) Book_Citings_List_New, so now takes 39 minutes. Further improvement required, given that Paper Summaries only takes 5 minutes.
- Books/Papers:
- Determined why the monthly regeneration process for Book-Paper Abstracts had doubled from 2.4 hours in January 2020 to 4.8 hours in April 2020. Successfully reduced to just upder one hour by materialising and indexing BookPaperAbstracts_List.
- Documenter:
- Notes:
- As revealed by Spider: Sundry uncategorised. Refs failing. 30 items. Fixed manually.
- Papers:
- Determined why the monthly regeneration process for Paper Summaries had ballooned from just over 3.5 hours in January 2020 to around 11.5 hours in April 2020. Fixed by materialising and indexing the view (now table) Paper_Citings_List_New, so now takes 6 minutes!
- Determined why the monthly regeneration process for Paper Abstracts had ballooned from just over 6 hours in January 2020 to over 17 hours in April 2020. Reduced to just under 4 hours as a result of materialising and indexing the view (now table) Paper_Citings_List_New. Further improvements required.
- Process:
- Wrote Check_Database_Size to check that the size of the database isn't at risk of breaching the 2GB limit. Required because of the need to re-materialise various views that have been instituted to speed up the full website regeneration routines, and which need to be run each time cmdRecalculate_Click is run, which is often.
- Full Website Regeneration took 60 hours in April 2020. This has now been substantially fixed, as of end June 2020, in that regeneration now takes just under 11 hours. While further improvements are possible, the best use of time is to avoid the need for regular regeneration altogether by completing my Cross-referencing project.
- Corrective work was split out into the items requiring improvement, must of which have now been implemented.
- History and analysis:-
- This 'balooning' was on my new laptop which has solid state disks - it was taking 36 hours on my old laptop, then halved when I got the replacement, but then nearly doubled!
- I investigated why this was so, and improved performance mainly by putting in a trace (using a timer - GetTickCount - found on the web that allows logging of elapsed time in milliseconds, using StartTimer and EndTimer) and finding the 'pinch points'
- I suspected that it was a new release of MS Access that's caused a problem; maybe a lost index or something similar. I couldn't find any evidence of this, but a mumber of queries were taking a second or so to execute, which small times multiplied up drastically when the number of pages on my site - over 100k - are taken into account. Quite why a cliff-edge had been reached, I know not!
- Three areas had been particularly impacted: Authors, Paper Abstracts & Paper Summaries, which have all trippled or worse. These are covered as seperate developments under 'Authors' and 'Papers'.
- Book Summaries and Book-Paper Abstracts had also at least doubled, but this has had less of an impact as they took much less time in the first place.
- Other regeneration processes - in particular 'Notes' - didn't appear to have been impacted.
- WebRefs:
- WebLinks_Tester_Full & WebLinks_Tester_Full_Map: Reformated Jump Table to 30 columns and multiple rows.
- As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 184 items. res://ieframe.dll/ in Returned_URL.
- WebLinks_Tester: If the URLs of WebRefs are changed between Spider runs, the URL mapping fails, and the WebRefs appear at the end of the last page, wrongly categorised. Fixed to ensure they appear on their own page, added to the jump table as 'WebRef Missing' and added an explanation.
- Amended WebRef 'Name' links to #Off-Page_Link_WxxxW style so referencing from other pages is possible (useful for Aeon)
- Webrefs_Update: Improved the processing of this sub which controls IE to check the URLs corresponding to WeRefs are still valid.
- Improved the recovery processing after IE fails or becomes detached. Now seems to work perfectly!
- Improved processing for 404 (page not found) errors - detect them where URL returned differs from that requested (so the error is correctly categorised) and also where it is the same (so the error is detected) - by using the GetElementsByTagName method to look through Title and H1 elemnts (where they exist).
- Note that there is still an issue for pdfs where the above methods don't work; though the page returned is an HTML or XML page, the changed URL still claims to be a pdf, so I can't check in case it's a real pdf and the process fails.
- Amend WebRef 'Name' link to #Off-Page_Link_WxxxW style. Also amend Quarterly and Annual task lists to use this reference.
- 20Q1
- Abstracts:
- As revealed by Spider: Link to PaperCitings failing (2 items) - Timing issue
- Authors:
- As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items. Dud page (Ontological Argument) deleted.
- Restrict jump-table at top of Authors pages to relevant items, and include the item counts.
- Bridge:
- As revealed by Spider: Bridge_Code_Archive.htm (14 items) & Bridge_Code_Archive_Recent.htm (5 items)
- Notes:
- Solve the problem of the size limitation for Notes by allowing their concatenation.
- As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
- As revealed by Spider: Blog20 Achived Notes - sundry links to Printable Notes. Live Note OK. 97 items.
- As revealed by Spider: Blackberry links from archived Note 1010 (Maintenance Dashboard). 38 items.
- As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
- Papers:
- Make use of Comments for large Abstracts seamless by appending Comments (if > 5k) to Abstracts.
- WebRefs:
- As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 51 items outstanding. res://ieframe.dll/ in Returned_URL.
- As revealed by Spider: WebLinks_Tester_4.htm. Refs failing. 32 items. res://ieframe.dll/ in Returned_URL.
- As revealed by Spider: WebLinks_Tester_Brief_12.htm (etc). Refs failing. 41 items. res://ieframe.dll/ in Returned_URL.
- WebRefs checker failing to detect broken links for Aeon - if a link fails, I get the message 'The page you were looking for could not be found (404)', but since the URL doesn't change, I don't spot it. This may happen for other sites as well. Fixed for Aeon - watch out for others.
- 19Q4
- Animadversions:
- Create Aeon_Files table & code to generate Aeon Page21 (to simplify processing and get round the 64k character limit).
- Authors:
- Improve author-citations process:-
- Show author-citation counts on Author Summary pages.
- Only have an author-citation link on Authors pages if the author is cited!
- Import Authors from Aeon_Files table.
- Notes:
- Create procedures (manual and via Functor_17) to add Audio Files to Notes
- Process:
- Fix Functors to enable previous quarter's Status Reports after Time period roll-over
- Full Website Regeneration failed in Notes Regen because the first Private Note (13322) couldn't be copied from the Transfer directory as the private note hadn't been created there by then.
- It should have been, from looking at the code.
- Also, the private note itself (116323) is not regenerated until later.
- In the start-December 2019 run, no such problem occurred. Maybe the problem was caused by copied between laptops. So, I've flagged it as complete for now, pending a recurrence.
- 19Q3
- Own Website
- Authors:
- As revealed by Spider: Author pages not regenerated when name corrected, leading to Sundry broken Links from other pages.
- Books/Papers:
- Print Write-up Notes (not just the links) for appropriate Papers in BookPaperAbstracts
- Bridge:
- NGS Refinements
- Add Mountnessing archive links to Results
- Synchronise Documentator with main website generator
- Infrastructure:
- Purchase & Commission new Lenovo Laptop: awaiting replacement following motherboard failure!
- PDFelement: Purchase & Commission upgrade from Version 5 to Version 7 for $39 - await new Laptop
- WebRefs:
- As revealed by Spider: Hits_Pages_Total_Crosstab_Descending.htm.
- Other Websites
- Sophie:
- 19Q2
- Authors:
- Added Jump Table to sections of Authors Pages
- General:
- Created 'Chess' Home Page
- Created as a main menu item and Note24
- Created consolidated results pages
- Added games & links to pgn4web
- Status:
- Corrected Quarterly Reporting for uneven Plan (Holidays)
- 19Q1
- Books/Papers:
- Removed Blackberry processing and associated pages as no longer needed with fast broadband.
- Photos:
- Timeline software: Added photos of Coxes Farm
- Created Photo-logging usage via Images table.
- Created Timeline software
- Technology:
- Investigated the Hit Stats provided by Webalizer and produced web-pages to show the most active pages.
- 18Q4
- Own Website
- Architecture:
- As revealed by Spider: Old links in unregenerated pages. 8 Items.
- Documenter:
- Outstanding Developments by Category appearing all in Bold, and links to lists failing.
- Notes:
- As revealed by Spider: Consciousness Status Report25 Achived Notes - links to BookCatalogCategorised_16_115_471.htm failing. Live Note OK. 46 items. 24 items outstanding.
- As revealed by Spider: Status Reports - Achived Notes - sundry links failing. 50 items in total. Live Notes fixed.
- Website: Maintenance Dashboard26. 14 items.
- Personal Identity Status Report27. 11 items.
- Summary Status Report28. 4 items.
- Actual Detail Summary Report29. 6 items.
- Chess Status Report30. 12 items.
- Etc. 3 items.
- As revealed by Spider: Sundry links failed for Supervisions. 30 items.
- As revealed by Spider: Links to sundry 'static' pages failing.
- 13 items fixed.
- 7 items outstanding: to do with Pete's PhD Test site. Links seem to be OK, so may be a problem with the Spider. Directory "Petes_PhD" was the only one set to "do not parse". The file wasn't in the site map. Reset; hopefully it'll work!
- As revealed by Spider: Links from Blog in Level 1 or 2 Printed Notes 51231 and 98132 failing. 141 items.
- Create Priority Task List Report33.
- Process:
- Spider:
- Determined why Full_Link_Up_Levels_Gen takes so long - approximately 4 hours. Re-architected to reduce database reads.
- Status:
- WebRefs:
- Added Display text to WebRefs for Links to Stanford, Wikipedia & Aeon
- Other Websites
- Enigma:
- Added 'EE' shortcut icon on all Webpage browser tabs
- Sophie:
- Added 'SB' shortcut icon on all Webpage browser tabs
- 18Q3
- Architecture:
- Added 'TT' shortcut icon on all Webpage browser tabs
- Authors:
- As revealed by Spider: 'Extra' Links to Notes from Citations List have the wrong directory (clumped in 1,000's like Papers rather than 100's). 7 items. Problem with AddCitations_List, so not just an issue for Authors.
- Authors Narrative: enable footnotes.
- Books:
- As revealed by Spider: BookCitings. 405 items. Fixed problem with CreateBookCitingsWebPages not checking for change of Book during 'Repeats' processing.
- Books/Papers:
- Paper & Book Abstracts - Make 'Disclaimer' more prominent & show 'Paper Statistics' rather than 'Paper Summaries'
- Bridge:
- Reformated + Fixed Hand-dealing pages to CSS standards
- Documenter:
- Created Functor using Form_Documentation_Links to automatically add control-links to documentation Notes.
- Fixed Bug where a variable has the same name as a procedure, and is thereby confused with one. Produced a warning query (Variable_Code_Name_Clashes) for manual correction.
- Footnotes:
- Notes:
- Created auto-regen of all Note Link (Note_nnn_Link) Pages
- As revealed by Spider: Notes_List_Control links fail for Supervisions. 58 items.
- As revealed by Spider: Religion Status Report39 Achived Notes - sundry links failing. Live Note OK. 59 items.
- As revealed by Spider: Languages Status Report40 Achived Notes - links to Dissertation failing. Live Note OK. 43 items.
- Blog: Created Functor to generate Blog from Blog.
- Created new '+LL+' Functor for links to Note_Link pages; fixed associated Notes.
- As revealed by Spider: Bridge Status Report41 Achived Notes - sundry links failing. Live Note OK. 70 items.
- As revealed by Spider: Links to Notes_xxx_Links, from Abstracts (94 items), Notes (25 … corrected manually), NotesPrint (1916). Fixed by +LL+ functor implementation.
- Automatic Key-word Hyperlinking to Notes Project: Phase 2
- Added links based on Book & Paper Titles
- Added ***Read***-marker to Lists
- Allowed links to be ignored (rather than suppressed) during approval process
- Skip links to Status Reports, etc.
- Photos:
- Status:
- 18Q2
- Architecture:
- Adjusted MS Access Control Page44 to feature new functions (Development Log & Automatic XRef) and consolidated the categorised Notes Lists into one button.
- Documenter:
- Investigated the error reports from the Documenter:
- Improved the diagostics
- Removed unused variables from the Generator code.
- General:
- Website Tidy-up:-
- Wrote a function to delete - with specified exceptions - items not regenerated in the last full website regeneration.
- Tidied up Site - deleted non-updated pages (especially old Printed Notes). Automatically on the local site, manually - using the audit log - on the live site.
- Investigated setting up an FTP link as a drive so I could automatically delete from the live site, but couldn't get it to work. Easy to set up as a drive, but not to assign a drive-letter so the drive can be used by VBA.
- Notes:
- Created automatic key-word referencing (ie. to Notes in the Notes-Group "Personal Identity"). This was complicated for a number of reasons:-
- There are always false positives, so there needed to be a way of "proofing" what's to be done before doing anything.
- There will be times where it is inappropriate (eg. in quoted work by other authors). So, there had to be a way of permanently suppressing such links.
- The referencing differs depending on whether the term is in a Footnote or not.
- I needed to check for already-existing links, variant grammatical forms, etc.
- I created a MsgBox routine - providing context - so that what needed to be done with potential links could easily be decided when the linking process was (re-)run.
- Links are always to Notes, but are from Notes, Archived Notes, Authors and Book & Paper Abstracts & Comments.
- Added a process to 'fix' - by adding the Note-ID - those note-links added without stating the ID. This will save processing time.
- Papers:
- Papers associated with broken links:-
- Created an area to hold pdfs, and automated links thereto.
- Created a pop-up to say I'd remove such pdfs if requested by their authors.
- Consequently, haven't password protected the area yet.
- Process:
- Re-engineered Website Maintenance Dashboard45 to show extra topical links (and show them at the top of the page).
- Status:
- Added extra items to Functors to facilitate Quarterly Reporting:-
- Automatically calculated QTD & YTD time statistics in the quarterly reports.
- Provided a YTD task list for Status Reports for projects "in abeyance" to make reporting easier.
- Modified the statistics - to be just YTD - for YTD Status reports.
- Created table for - by Project - "Plans for the near future" tasks, and a functor to add them to the status report.
- Took "items competed" and "plans for the near future from the development log for the Web-tools status report.
- Created another functor to add these lists to the Summary report.
- WebRefs:
- 18Q1
- Bridge:
- Corrected format of Links from Bridge Page
- Documenter:
- Documenter Bug-fixes: Objects in Comments, MsgBoxs & Debugs
- Created consolidated Development Log, and updated "Progress to Date" and "Outstanding Developments" Reports
- Infrastructure:
- Migration of Website from UKServers to Namesco. Test out.
- Notes:
- Papers:
- Correction of narratives on Note time-recording pseudo-papers
- Process:
- Improved efficiency of daily changes regeneration (removed msgboxes)
- Wrote a routine to update a set of .htm files, replacing text in them all
- Status:
- Created web-page generator for Development items (ie. Created the Web-Project "To Do" List and "Outstanding Developments" directly from this list).
- WebRefs:
- Documentation & Bug-fixes: Phase 1
- Completed "as is" functional documentation so I could see what's going on.
- Re-engineered so it made sense and ran efficiently without bugs
- Fixed the obvious data errors that confused the statistics - ie. where the returned URL differs from that requested and https issues.
- Wrote / improved diagnostic reports (web pages)
- Determined & reported usage directly using embedded +WnnnW+ encoding rather than via the Spider. Reconciled the two methods.
- 17Q4
- Architecture:
- Automated links between Quarterly reports (Task Lists)
- Books/Papers:
- New "Recent Acquisitions" Book List - Uncategorised
- Documenter:
- Code Archive: Hadn't been working - fixed and recovered a selection from backups
- Code Archive: Created Web-pages of Recent Code Changes
- Notes:
- Added %age complete to Notes Reading Lists
- Enabled deduction of Note link by omitting the Note ID from the mark-up and doing a look up on the key-word and its varients.
- Timesheets:
- Separated Time Recording for Aeon
- 17Q3
- Notes:
- Added Colour-coded "quality markers" to Notes: included them in lists and jump-tables
- 17Q2
- Bridge:
- Created hyperlinks from my consolidated results pages to the actual results
- Search:
- Wrote routines to create a cut-down Website excluding pages with irrelevant or duplicated information. Used this to create the ZoomSearch facility.
- 17Q1
- Bridge:
- Added new features (70s Club & Monthly Pairs Winners) to my Bridge mini-websites.
- Created Master Point Lists & Promotions for the ECBA website.
- 16Q4
- Spider:
- Added “.shtml” and “.shtm” to the list of web-suffices.
- 16Q3
- Acts:
- Developed a website for displaying the textual and grammatical analyses and appendices of Pete’s PhD on the Acts of the Apostles. Two versions: Live (Acts: Live Site (Defunct)) and Test (Acts: Test Site).
- Bridge:
- 16Q2
- Clock:
- Created a "chess clock" application to help time tasks. Not much use in the end.
- 16Q1
- Own Website
- Books/Papers:
- Created direct links from Subject, Topic & Sub-topic (in Book and Paper Summaries) to their link pages.
- Documenter:
- Created an automatic Forms documenter – with links to the code.
- Notes:
- Stopped (I hope!) “private” Notes showing in Paper Abstracts.
- Completed converting “static” pages to Notes.
- Enabled direct links to Notes from within Footnotes.
- Allowed Note- & Footnote-links from multiple words54. This is such a useful feature!
- WebRefs:
- Some tidying up of the 3,000-odd external links from my website.
- Other Websites
- Bridge:
- Reconciled my X-IMP calculations with a new feature provided by Bridgewebs.
- Accommodated Alaric’s new scoring system for St. Edith’s.
- Created Ladies’ and Gentlemen’s ladders for my mini-websites.
- 15Q4
- Own Website
- Abstracts:
- Corrected data format on Identity Abstracts
- Aesthetics:
- Reduced font + changed text in ubiquitous THEO'S PAGE heading
- Authors:
- Corrected author's names (SN (FN)) for all papers and books
- Provided easy links to works by the same author; Books, Papers; Joint authors
- Backups:
- Enhanced my backup system to prune items backed up multiple times (eg. if they moved directory).
- Books/Papers:
- Updated booklists to include shelf-identifiers
- Neatened up the linkage headers in Book & Paper Abstracts.
- Data:
- Corrected data content on Identity Sub-topics
- Documenter:
- Segregated the code-archive, which was bloating55 the generator database.
- Notes:
- Fixed problem of images not appearing in "Printed" Notes (wrong directory level) unless the images are repeated
- Converted the “static” pages in my website to Note format (so they can change and retain history).
- Process:
- Created a consolidated processor to enable overnight regeneration of the whole website.
- Spider:
- Finally cracked the Spider for external links, and used it to check the 3,000-odd links from my website. Some tidying up outstanding.
- Other Websites
- Bridge:
- Segregated the Bridge website generators from those for my own website. They now live in separate databases and have their own documenters.
- 15Q3
- Own Website
- Architecture:
- Made a lot of progress on improving the hyperlinking in my own website – I’ve made a good start on this project. The links now work accurately, but I need to ensure the efficient propagation of linkage changes across the site without requiring a complete regeneration.
- Notes:
- Tidied up the archived Notes.
- Other Websites
- Bridge:
- Created a new website (using Bridgewebs) for the First Class Bridge Academy (First Class Bridge).
- Ported my “handicap / improvers” routines from Hutton to “Mini Websites”.
- Enigma:
- Created a small website for a music group Julie and I attend – the Enigma Ensemble (Defunct).
- Sophie:
- 15Q2
- Bridge:
- Development of the “Eve Goblets” (victor ludorum) software for the ECBA, and roll-over to the new season.
- 15Q1
- Own Website
- Architecture:
- Vastly improved the “bulleting” options in Notes, etc.
- Authors:
- Improved the format and content of “Authors” pages.
- Timesheets:
- Segregated time-recording so I can easily see how much time is spent developing and supporting my own website, and how much is spent on those of other organisations.
- Other Websites
- Bridge:
- Further improvements to my Butler-IMPs and Cross-IMPs calculation routines.
- Started a project to provide small websites for bridge clubs without sites: Essex Bridge Results.
- Took on the ECBA Website (Essex Contract Bridge Association), at the cost of considerable effort by way of reformatting & tidy-up.
- 14Q4
- Own Website
- Notes:
- Made improvements to the regeneration and referencing of archived Notes.
- Spider:
- Fixed the bug wherein the Spider ran out of system resources on large runs: now compacts & repairs the slave database as it does along (every 200k links; parameterised).
- Other Websites
- Bridge:
- Improvements to my Butler-IMPs and Cross-IMPs calculation routines – different formats, including the Jeff Smith format, and extension to Butler-scored Pairs events. This work has been taken up by the ECBA selectors for county-team selection.
- 14Q3
- Own Website
- Notes:
- Started to split Paper Abstracts from Write-ups.
- Created a progress-tracker57 for my Thesis.
- Improved error-recovery where encoding fails for Notes links.
- Spider:
- Continued working on my Spider and started to use it to clean up my website by diagnosing broken links.
- Other Websites
- Bridge:
- Created a suite of programs to extract data and calculate Butler-IMPs and Cross-IMPs from the travellers on Bridgewebs Teams events, and output ranking-lists to the Web.
- 14Q2
- Architecture:
- Added extra linkages between pages (eg. on Author).
- Authors:
- Created the facility for Author Narratives.
- Backups:
- Used a clone of my Spider routines to create a backup and restore system.
- Documenter:
- Created multiple secure areas on my website, so that Documentation is no longer hiding under my Supervision Notes.
- Notes:
- Wrote routines to import old (but missing) archived Notes into the database so they can be (and have been) re-exported in the current format.
- 14Q1
- Architecture:
- Repaired / updated my Website site-map
- Notes:
- Made a lot of improvements to my Site generators, in particular the pesky bug that stopped the links to archived Notes working correctly.
- Process:
- Improved the regeneration routines for the various classes of webpage, and got them to record statistics in support of the “Website Regeneration Dashboard”.
- Created a “Website Regeneration Dashboard” to show when the various elements of my website have been refreshed, and how long the routines took to run. Links to this appear in the footer to each webpage.
- 13Q3
- Architecture:
- Sorted conflicts between Subject, Topic & Sub-Topic IDs
- Books/Papers:
- Improved linkages between Book/Paper Lists and Summaries or Abstracts
- Made enhancements to "Books & Papers Collection Categorised" Pages
- Notes:
- Simplified / Re-engineered Notes processing
- "Single Image" development for Notes
- Allowed "Authors Works Lists" to be linked-to from Notes
- Created Archived Notes Re-Export facility
- Status:
- Re-engineered Monthly Reporting on Change of Year
- Technology:
- Used CSS in all manually-crafted and automatically-generated pages
- Converted Tables to CSS3 standard
- Timesheets:
- Re-engineered Time-Recording, so that actuals are held against Papers (and Books) in a separate table, rather than as columns by year in the Papers / Books tables themselves. Retained the grand totals only.
- 13Q2
- Own Website
- Architecture:
- Replaced macros with code.
- Improved error-recovery where encoding fails for links to Books, Papers and Authors. That is, where one of my bespoke tags (eg. +B58) doesn’t match with its corresponding anti-tag (eg. B+).
- Books/Papers:
- Developed "Books by Location" Pages
- Improved the linkage between Papers and Books.
- Bridge:
- Created “Consolidated Bridge Results” webpages (for my own results).
- Documenter:
- Wrote the Code Documenter.
- Created a code & query archive facility as part of the Code Documenter.
- Notes:
- Improved "Concatenated Notes" production
- Photos:
- Reduced the size of photos (to make the pages quicker to load).
- Spider:
- Status:
- Improved Quarterly Reporting
- Improved "This Month's Summary" Reporting
- WebRefs:
- Some pages render oddly in Chrome (and maybe other browsers) – investigated59.
- Other Websites
- Bridge:
- Merged60 Philosophy & Bridge Databases
- 13Q1
- Status:
- Re-engineered Monthly Reporting Tables & improved Monthly Reporting
- Technology:
- 12Q3
- General:
- Improved Titles for Search Results display optimisation
- 11Q4
- Books/Papers:
- Showed Percentage of Paper / Book Read
- 11Q2
- Authors:
- Created Author Cross-reference
- Books/Papers:
- Added “changed dates” to Paper and Book abstracts, and produced “recently changed” pages for them
- Created Inter-Book-Paper Citings Cross-reference
- 10Q3
- Notes:
- Wrote routines to automatically generate the “Printable Notes” footer – and to create the corresponding web pages – from the control table. Very useful.
- Investigate Note_Links: needs versioning, and section of calling Note
- Add “Date Created” to Note_Links (for audit purposes; I’m still not convinced that links disappear when they should).
- WebRefs:
- Improved the functionality for automatically generating external Web-links in Notes, Book Abstracts and Paper Abstracts. Corrected current data to accommodate the new code.
- 10Q2
- Footnotes:
- Added intra-Note footnotes – a real breakthrough (strange I’d not thought of this before). It will enable me to convert to Note form all the old .pdfs from my undergraduate days – which are replete with footnotes – if this is any use.
- Spider:
- Wrote my own Web Spider.
- It’s a little bit of a cheat, in that it doesn’t actually invoke the web-links, but crawls through my directory structure selecting all the .htm files, and rummages through the HTML looking for links which it then copies to a table.
- It also maintains a table of the directory structure and files therein, and then works out the real address from the relative addressing.
- It then allows cross-checking between the links and the files.
- Links external to the site have to be investigated manually (at least I have a list of all the links) – unless I can work out how to use the list to power a browser to check.
- There are about 333k links in 55k files – the full run took about 8 hours.
- It now runs incrementally based on file last-updated dates.
- 09Q4
- Notes:
- Restricted the size of the “previous versions” tables for Notes to 12.
- Process:
- Set up my database system to roll forward to the next academic year, and generate web-pages appropriately.
- 09Q3
- Books/Papers:
- Made a lot of changes to the way Books and Papers are categorised; in particular:-
- Added 2-level jump tables, to save the browser being crowded with hundreds of cross-categorisations in the same page.
- While it’s OK for Books, it seems to have a few potholes for Papers, as some of the links fail.
- The problem may be something to do with my economical way of suppressing redundant sub-categorisation - ie. where one or more of Subject, Topic and Sub-topic are the same.
- Completed a full 2-way link from Book/Paper Summaries to Notes.
- Notes:
- Functionally enhanced the Notes-generation processing options; to make niftier production possible. In particular, where individual Notes are selected, I made the re-generation of the cross-references optional (so that if only textual variation is being iterated, the iterations are quicker).
- Enabled automatic immediate promotion to Live Website of selected Temp Notes.
- Enabled Concatenated Notes (by Group). This is only allowed for “Identity” Notes at the moment, which is what I’d wanted it for. It allows all Notes for a Group to be printed off in Title sequence, for ease of review.
- Ensured that the Notes Reading-list is generated from all 3 sub-topics.
- Improved the performance of “All Notes” regeneration. The problem was with the generation of the generic reading lists based on Note Title. These only apply to the “Identity” Notes, and it was taking a long time to determine there was no list to produce.
- Process:
- Created a “performance stats” database and routines to write to it – though I only implemented them for the Notes generator.
- 09Q2
- Architecture:
- Created Site-map for Website
- Books/Papers:
- Added "quality marker" to Paper & Book Abstracts
- Improved, and corrected errors in, the Blackberry-friendly formatted lists of Books and Papers.
- De-duplicated Books / Papers to Notes links
- Notes:
- Added a table of Notes history at the end of each Note (ie. not just previous version) - to save tedious chaining
- 09Q1
- Books/Papers:
- Enabled links from Books to the Papers contained therein
- Provided an automated link from Books / Papers to the Notes referencing them
- Enhanced Books to Papers links to include papers multiply held
- Created automatic links from Paper Summary to Book Summary
- Improved the Books / Papers to Notes links to include the right Version and Section (and historical Title) of the Note
- Notes:
- Created optional "don't print" feature for specific footnotes.
- Fixed the problem of the overpopulation of the Notes directory
- Added reading-list (of works cited) at the end of the Note, as well as in the Printed Note
- Option for Notes-Regen:
- Changed-only
- Changed, and all impacted by Change
- All
- 08Q4
- Timesheets:
- Updated Book totals from totals of the Papers contained therein
- 08Q1
- Notes:
- Allowed regeneration of Temp Notes only
- 07Q4
- Architecture:
- Added password-protection to selected pages
- Notes:
- Added referenced reading lists to printable version of Notes
- Added referenced Books / Papers as footnote jump table to Notes
- Improve "Temp" note processing. Suppress "_T" suffix (just don't copy to Transfer)
- When a Note is archived, ensured a printable note is also archived62 (if this Note has a print)
- Search:
- Created search facility using ZoomSearch
- 07Q3
- Animadversions:
- Added a "hot topics" link - things that I'm currently working on and want feedback ("Animadversions")
- Footnotes:
- Improved transparency of Footnote referencing in printed Notes
- Notes:
- Created "respondent notes" - database identifier & red indicator + text
- Enabled generation of printable version of Notes
- Created63 a "quick lookup" table of responses to a note and its sub-notes (recursive function on Notes_Links)
- Made Note-change-history available on-line (added status + publishable flags)
- 07Q2
- Abstracts:
- Segregated Abstracts into multiple directories
- Created auto-backup of Abstracts, and more efficient maintenance web-routines
- Added "Extra Data" parallel to Abstracts; ie. "More …" (Location etc.)
- Architecture:
- Segregated database64 into Master and Slaves (code, papers, books, notes, web control, … vs PI)
- Tidied up Footers throughout Philosophy Papers site (made into tables)
- Books/Papers:
- Created "Papers by Title within Topic" list
- Created "Identity Read Papers by Sub-Topic" list
- Added Flags for "laptop" and "photocopy"; removed document location from Comments
- Added Book Comments and Abstracts (as "More…") to Book-lists
- Created "BB-friendly65" lists: Papers (Full) and Books
- Added multiple Jump-levels for BB Lists (to keep pages small)
- Added change of author or title in "Abstracts audit"
- Created "Identity Papers Full by Sub-Topic" list
- Jump-Tables: added "by topic" option
- Added "read" column in papers / books lists
- Footnotes:
- Created auto-renumbering66 routine for Footnotes
- Created auto-backup of Footnotes67, and more efficient maintenance web-routines
- General:
- Improved "Disclaimer68" Page. Reduce font, add ref to philosophy databases + hyperlinks
- Notes:
- Added "footnote" pages69 for Carthusians & King's
- Added "outgoing links" summary footer to Notes pages
- Allowed two sorts of Note - document-specific ("Write-up") and general
- Process:
- Improved ease of FTP process by use of Transfer Directory
- WebRefs:
- Made Web Refs in Comments / Abstracts into Hyperlinks
Items Completed by Category:
- Own Website: Completed Items By Category:-
- Abstracts
- 20Q1
- As revealed by Spider: Link to PaperCitings failing (2 items) - Timing issue
- 15Q4
- Corrected data format on Identity Abstracts
- 07Q2
- Added "Extra Data" parallel to Abstracts; ie. "More …" (Location etc.)
- Created auto-backup of Abstracts, and more efficient maintenance web-routines
- Segregated Abstracts into multiple directories
- Aesthetics
- 15Q4
- Reduced font + changed text in ubiquitous THEO'S PAGE heading
- Animadversions
- 19Q4
- Create Aeon_Files table & code to generate Aeon Page70 (to simplify processing and get round the 64k character limit).
- 07Q3
- Added a "hot topics" link - things that I'm currently working on and want feedback ("Animadversions")
- Architecture
- 21Q2
- Created Check_Delimiters to protect against unpaired delimiters in lists, which squiff the formatting of documents that contain them. Output diagnostics to MsgBox, if not a re-gen, and to debug window to enable correction.
- 20Q3
- 20Q2
- Enhanced the Functor processing to allow parameters. Previously, I had to create a new Functor ID each time for a very similar requirement. The main problem with this old method was that these Functors (where they call another Functor, or subroutine from a row in Functor) didn't feed through into the Documenter subroutines properly.
- Truncated Cross_Reference_Changes table (from over 1M rows to under 50k) which should be self-maintaining, but isn't. This seems to have improved the regeneration times of Notes, Archived Notes and some other objects now that insertions into this table are quicker. Needs a permanent solution as part of my Cross-Referencing project.
- 18Q4
- As revealed by Spider: Old links in unregenerated pages. 8 Items.
- 18Q3
- Added 'TT' shortcut icon on all Webpage browser tabs
- 18Q2
- Adjusted MS Access Control Page71 to feature new functions (Development Log & Automatic XRef) and consolidated the categorised Notes Lists into one button.
- 17Q4
- Automated links between Quarterly reports (Task Lists)
- 15Q3
- Made a lot of progress on improving the hyperlinking in my own website – I’ve made a good start on this project. The links now work accurately, but I need to ensure the efficient propagation of linkage changes across the site without requiring a complete regeneration.
- 15Q1
- Vastly improved the “bulleting” options in Notes, etc.
- 14Q2
- Added extra linkages between pages (eg. on Author).
- 14Q1
- Repaired / updated my Website site-map
- 13Q3
- Sorted conflicts between Subject, Topic & Sub-Topic IDs
- 13Q2
- Improved error-recovery where encoding fails for links to Books, Papers and Authors. That is, where one of my bespoke tags (eg. +B72) doesn’t match with its corresponding anti-tag (eg. B+).
- Replaced macros with code.
- 09Q2
- Created Site-map for Website
- 07Q4
- Added password-protection to selected pages
- 07Q2
- Segregated database73 into Master and Slaves (code, papers, books, notes, web control, … vs PI)
- Tidied up Footers throughout Philosophy Papers site (made into tables)
- Authors
- 20Q2
- Determined why the monthly regeneration process for Authors has ballooned from under 4 hours in January 2020 to over 17 hours in April 2020. Fix on 11/04/20 - time more than halved to 8.25 hours - by re-engineering the 'menu' process that counts the items linked to further down the page.
- Enabled "alternate names" for identically-named authors (to avoid middle-initial = X):-
- Set up and populated Author_Name_Display
- Ensured Author_Name_Display appears on the Author page.
- Further reduced the monthly regeneration process time for Authors from 8.25 hours to 13 minutes! Fixed by materialising and indexing a 'view' to table Authors_Cited_By_All_List.
- 20Q1
- As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items. Dud page (Ontological Argument) deleted.
- Restrict jump-table at top of Authors pages to relevant items, and include the item counts.
- 19Q4
- Improve author-citations process:-
- Show author-citation counts on Author Summary pages.
- Only have an author-citation link on Authors pages if the author is cited!
- Import Authors from Aeon_Files table.
- 19Q3
- As revealed by Spider: Author pages not regenerated when name corrected, leading to Sundry broken Links from other pages.
- 19Q2
- Added Jump Table to sections of Authors Pages
- 18Q3
- As revealed by Spider: 'Extra' Links to Notes from Citations List have the wrong directory (clumped in 1,000's like Papers rather than 100's). 7 items. Problem with AddCitations_List, so not just an issue for Authors.
- Authors Narrative: enable footnotes.
- 15Q4
- Corrected author's names (SN (FN)) for all papers and books
- Provided easy links to works by the same author; Books, Papers; Joint authors
- 15Q1
- Improved the format and content of “Authors” pages.
- 14Q2
- Created the facility for Author Narratives.
- 11Q2
- Created Author Cross-reference
- Backups
- 15Q4
- Enhanced my backup system to prune items backed up multiple times (eg. if they moved directory).
- 14Q2
- Used a clone of my Spider routines to create a backup and restore system.
- Books
- 20Q3
- BookCitings references not sequenced correctly. Corrected.
- BooksToNotes Link Pages: Re-engineered - used CreatePapersToNotesWebPages to generate pages
- 20Q2
- Determined why the monthly regeneration process for Book Summaries had ballooned from 1.1 hours in January 2020 to 3.5 hours in April 2020. Partly fixed by by materialising and indexing the view (now table) Book_Citings_List_New, so now takes 39 minutes. Further improvement required, given that Paper Summaries only takes 5 minutes.
- 18Q3
- As revealed by Spider: BookCitings. 405 items. Fixed problem with CreateBookCitingsWebPages not checking for change of Book during 'Repeats' processing.
- Books/Papers
- 20Q4
- 20Q3
- 20Q2
- Determined why the monthly regeneration process for Book-Paper Abstracts had doubled from 2.4 hours in January 2020 to 4.8 hours in April 2020. Successfully reduced to just upder one hour by materialising and indexing BookPaperAbstracts_List.
- 19Q3
- Print Write-up Notes (not just the links) for appropriate Papers in BookPaperAbstracts
- 19Q1
- Removed Blackberry processing and associated pages as no longer needed with fast broadband.
- 18Q3
- Paper & Book Abstracts - Make 'Disclaimer' more prominent & show 'Paper Statistics' rather than 'Paper Summaries'
- 17Q4
- New "Recent Acquisitions" Book List - Uncategorised
- 16Q1
- Created direct links from Subject, Topic & Sub-topic (in Book and Paper Summaries) to their link pages.
- 15Q4
- Neatened up the linkage headers in Book & Paper Abstracts.
- Updated booklists to include shelf-identifiers
- 13Q3
- Improved linkages between Book/Paper Lists and Summaries or Abstracts
- Made enhancements to "Books & Papers Collection Categorised" Pages
- 13Q2
- Developed "Books by Location" Pages
- Improved the linkage between Papers and Books.
- 11Q4
- Showed Percentage of Paper / Book Read
- 11Q2
- Added “changed dates” to Paper and Book abstracts, and produced “recently changed” pages for them
- Created Inter-Book-Paper Citings Cross-reference
- 09Q3
- Completed a full 2-way link from Book/Paper Summaries to Notes.
- Made a lot of changes to the way Books and Papers are categorised; in particular:-
- Added 2-level jump tables, to save the browser being crowded with hundreds of cross-categorisations in the same page.
- While it’s OK for Books, it seems to have a few potholes for Papers, as some of the links fail.
- The problem may be something to do with my economical way of suppressing redundant sub-categorisation - ie. where one or more of Subject, Topic and Sub-topic are the same.
- 09Q2
- Added "quality marker" to Paper & Book Abstracts
- De-duplicated Books / Papers to Notes links
- Improved, and corrected errors in, the Blackberry-friendly formatted lists of Books and Papers.
- 09Q1
- Created automatic links from Paper Summary to Book Summary
- Enabled links from Books to the Papers contained therein
- Enhanced Books to Papers links to include papers multiply held
- Improved the Books / Papers to Notes links to include the right Version and Section (and historical Title) of the Note
- Provided an automated link from Books / Papers to the Notes referencing them
- 07Q2
- Added "read" column in papers / books lists
- Added Book Comments and Abstracts (as "More…") to Book-lists
- Added change of author or title in "Abstracts audit"
- Added Flags for "laptop" and "photocopy"; removed document location from Comments
- Added multiple Jump-levels for BB Lists (to keep pages small)
- Created "BB-friendly74" lists: Papers (Full) and Books
- Created "Identity Papers Full by Sub-Topic" list
- Created "Identity Read Papers by Sub-Topic" list
- Created "Papers by Title within Topic" list
- Jump-Tables: added "by topic" option
- Bridge
- 20Q1
- As revealed by Spider: Bridge_Code_Archive.htm (14 items) & Bridge_Code_Archive_Recent.htm (5 items)
- 19Q3
- Add Mountnessing archive links to Results
- NGS Refinements
- Synchronise Documentator with main website generator
- 18Q3
- Reformated + Fixed Hand-dealing pages to CSS standards
- 18Q1
- Corrected format of Links from Bridge Page
- 17Q2
- Created hyperlinks from my consolidated results pages to the actual results
- 13Q2
- Created “Consolidated Bridge Results” webpages (for my own results).
- Clock
- 16Q2
- Created a "chess clock" application to help time tasks. Not much use in the end.
- Data
- 15Q4
- Corrected data content on Identity Sub-topics
- Documenter
- 20Q2
- 18Q4
- Outstanding Developments by Category appearing all in Bold, and links to lists failing.
- 18Q3
- Created Functor using Form_Documentation_Links to automatically add control-links to documentation Notes.
- Fixed Bug where a variable has the same name as a procedure, and is thereby confused with one. Produced a warning query (Variable_Code_Name_Clashes) for manual correction.
- 18Q2
- Investigated the error reports from the Documenter:
- Improved the diagostics
- Removed unused variables from the Generator code.
- 18Q1
- Created consolidated Development Log, and updated "Progress to Date" and "Outstanding Developments" Reports
- Documenter Bug-fixes: Objects in Comments, MsgBoxs & Debugs
- 17Q4
- Code Archive: Created Web-pages of Recent Code Changes
- Code Archive: Hadn't been working - fixed and recovered a selection from backups
- 16Q1
- Created an automatic Forms documenter – with links to the code.
- 15Q4
- Segregated the code-archive, which was bloating76 the generator database.
- 14Q2
- Created multiple secure areas on my website, so that Documentation is no longer hiding under my Supervision Notes.
- 13Q2
- Created a code & query archive facility as part of the Code Documenter.
- Wrote the Code Documenter.
- Footnotes
- 18Q3
- 10Q2
- Added intra-Note footnotes – a real breakthrough (strange I’d not thought of this before). It will enable me to convert to Note form all the old .pdfs from my undergraduate days – which are replete with footnotes – if this is any use.
- 07Q3
- Improved transparency of Footnote referencing in printed Notes
- 07Q2
- Created auto-backup of Footnotes79, and more efficient maintenance web-routines
- Created auto-renumbering80 routine for Footnotes
- General
- 19Q2
- Created 'Chess' Home Page
- Created as a main menu item and Note81
- Created consolidated results pages
- Added games & links to pgn4web
- 18Q2
- Website Tidy-up:-
- Wrote a function to delete - with specified exceptions - items not regenerated in the last full website regeneration.
- Tidied up Site - deleted non-updated pages (especially old Printed Notes). Automatically on the local site, manually - using the audit log - on the live site.
- Investigated setting up an FTP link as a drive so I could automatically delete from the live site, but couldn't get it to work. Easy to set up as a drive, but not to assign a drive-letter so the drive can be used by VBA.
- 12Q3
- Improved Titles for Search Results display optimisation
- 07Q2
- Improved "Disclaimer82" Page. Reduce font, add ref to philosophy databases + hyperlinks
- Infrastructure
- 19Q3
- PDFelement: Purchase & Commission upgrade from Version 5 to Version 7 for $39 - await new Laptop
- Purchase & Commission new Lenovo Laptop: awaiting replacement following motherboard failure!
- 18Q1
- Migration of Website from UKServers to Namesco. Test out.
- Notes
- 21Q1
- Developed a mechanism to make the reading lists in my PID Notes easier to maintain and transfer to the Chapters on my Thesis:-
- Created a new table to hold the reading lists (include some static so it's easy to maintain): PID_Note_Reading_Lists
- Created a new Functor to use this table to generate the reading lists within a PID Note (automatically segregating "read" from "unread": allowed to run with two options to allow for comments between the lists: Functor_22, Options 6 and 7, respectively).
- Enabled footnotes for reading-list items.
- Created one-off queries (PID_Notes_Books_Referenced_Gen & PID_Notes_Papers_Referenced_Gen) to extract the reading lists from PID Notes and populate the new table (using the Cross_Reference table)
- To prove the process, started manually to replace the Reading Lists in all the PID Notes with the Functors (completing this will be a manual exercise and, after each run, will involve checking Live versus Test sites).
- Procedurally, in the Thesis Chapters, if Notes are referenced in the Text, use the "NP" encoding so that they don't contaminate the reading lists: only those explicitly listed should do so.
- Added flags to either exclude Books or Papers logged in PID_Note_Reading_Lists altogether, or to segregate them as "missed the cut" in the Thesis Chapters (and, possibly) the Notes themselves.
- Automatically update table PID_Note_Reading_Lists to maintain the "read status" and Book/Paper Title.
- Developed a mechanism to monitor usage of PID Notes within my Thesis (by Chapter):-
- Created Note 131783, which incorporates as table and a list, created as follows ...
- Used Notes: Functor_21, option 14 (Query PID_Notes_Used_By_Thesis_Chapter_Xtab)
- Unused Notes: Functor_22, option 5 (Query PID_Notes_Unused_By_Thesis)
- Enabled multiple parameters for Functors
- Enhanced Functors to work for selected non-temp Notes (ie. PID Notes: ensured Functor_22, Options 6 and 7 can run on a non-Temp Note, without creating an Archived Note; borrowed code from Auto_Reference_Notes) so that up-to-date stats can be incorporated. This was fairly complex as I wanted to avoid production of an Archived Note each time a non-Temp Note is regenerated. I also wanted to ensure that Notes whose variable text consists entirely of Functors get archived correctly (which they might not have if I'd removed Functor-generated text from Notes before saving them to the Notes table; in fact, the text saved to Live and Archive is identical and includes the full Functor content).
- Enhanced the mechanism to make the reading lists in my PID Notes easier to maintain:-
- Made the table PID_Note_Reading_Lists easier to maintain: added a menu at the start of the Create Notes Web Pages84 to (inter alia) open the PID_Note_Reading_Lists table, add or delete rows, change the "excluded" and "cut" flags, and amend Footnotes. Book/Paper titles and read-status are automatically maintained by the Functors.
- Added functionality to allow the reading lists to be further segregated - as in the Notes Animalists85 and Christian materialism86 - which din't use the Functors (though their reading lists are on PID_Note_Reading_Lists).
- Created a further Functor (Functor_23, Option 17) to generate the Thesis Chapter Reading Lists from the Notes explicitly listed as relevant in the new table Thesis_Note_XRef. Uses Functor_Indented_List, which also creates (via Functor_23, Option 16) the indented PID-Note lists. A major triumph!
- Modified Functor_21 Option 3 to allow multiple Audio Files per timesheet line
- 20Q4
- 20Q3
- Added Aeon Abstract (ie. Footnote) link to Aeon Webref items in Summary task List reports for 'read' items
- Added option in Auto_Reference_Notes to - for Live (ie. not TEMP) Notes - update the latest Archived Note to equal the updated Live Note, hence avoiding superfluous archives.
- Added option in Auto_Reference_Notes to only confirm new items (leaving previously-flagged items untouched)
- Corrected Issues with Nested Functors: Added Functor_ID to End-tag
- Determined why the run of Auto_Reference_Notes_Regen took 2.5 hours on 02/09/2020 (and climbing). It was down to the Aeon Note90 being searched; in particular Check_If_In_Container, checking for items in Footnotes and / or Quoted Text. As this Note is regenerated, there's no point auto-linking it. Time reduced to 23 minutes once this Note - and other Notes of inappropriate Note Groups - was removed.
- Highlighted Archived Notes clearly as not the latest
- Upgraded Auto_Reference_Notes to allow a parameterised 'termination or continue' for very long update runs
- Upgraded Auto_Reference_Notes to avoid key-words within +AA+-style Authors names (using Check_If_In_Container).
- Upgraded Auto_Reference_Notes to log its actions to a new table (Auto_Reference_Notes_Actions) so that any errors can be investigated and improvements made.
- 20Q2
- As revealed by Spider: Sundry uncategorised. Refs failing. 30 items. Fixed manually.
- 20Q1
- As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
- As revealed by Spider: Blackberry links from archived Note 1010 (Maintenance Dashboard). 38 items.
- As revealed by Spider: Blog91 Achived Notes - sundry links to Printable Notes. Live Note OK. 97 items.
- As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
- Solve the problem of the size limitation for Notes by allowing their concatenation.
- 19Q4
- Create procedures (manual and via Functor_17) to add Audio Files to Notes
- 18Q4
- As revealed by Spider: Consciousness Status Report92 Achived Notes - links to BookCatalogCategorised_16_115_471.htm failing. Live Note OK. 46 items. 24 items outstanding.
- As revealed by Spider: Links from Blog in Level 1 or 2 Printed Notes 51293 and 98194 failing. 141 items.
- As revealed by Spider: Links to sundry 'static' pages failing.
- 13 items fixed.
- 7 items outstanding: to do with Pete's PhD Test site. Links seem to be OK, so may be a problem with the Spider. Directory "Petes_PhD" was the only one set to "do not parse". The file wasn't in the site map. Reset; hopefully it'll work!
- As revealed by Spider: Status Reports - Achived Notes - sundry links failing. 50 items in total. Live Notes fixed.
- Website: Maintenance Dashboard95. 14 items.
- Personal Identity Status Report96. 11 items.
- Summary Status Report97. 4 items.
- Actual Detail Summary Report98. 6 items.
- Chess Status Report99. 12 items.
- Etc. 3 items.
- As revealed by Spider: Sundry links failed for Supervisions. 30 items.
- Create Priority Task List Report100.
- 18Q3
- As revealed by Spider: Bridge Status Report101 Achived Notes - sundry links failing. Live Note OK. 70 items.
- As revealed by Spider: Languages Status Report102 Achived Notes - links to Dissertation failing. Live Note OK. 43 items.
- As revealed by Spider: Religion Status Report103 Achived Notes - sundry links failing. Live Note OK. 59 items.
- As revealed by Spider: Links to Notes_xxx_Links, from Abstracts (94 items), Notes (25 … corrected manually), NotesPrint (1916). Fixed by +LL+ functor implementation.
- As revealed by Spider: Notes_List_Control links fail for Supervisions. 58 items.
- Automatic Key-word Hyperlinking to Notes Project: Phase 2
- Added links based on Book & Paper Titles
- Added ***Read***-marker to Lists
- Allowed links to be ignored (rather than suppressed) during approval process
- Skip links to Status Reports, etc.
- Blog: Created Functor to generate Blog from Blog.
- Created auto-regen of all Note Link (Note_nnn_Link) Pages
- Created new '+LL+' Functor for links to Note_Link pages; fixed associated Notes.
- 18Q2
- Added a process to 'fix' - by adding the Note-ID - those note-links added without stating the ID. This will save processing time.
- Created automatic key-word referencing (ie. to Notes in the Notes-Group "Personal Identity"). This was complicated for a number of reasons:-
- There are always false positives, so there needed to be a way of "proofing" what's to be done before doing anything.
- There will be times where it is inappropriate (eg. in quoted work by other authors). So, there had to be a way of permanently suppressing such links.
- The referencing differs depending on whether the term is in a Footnote or not.
- I needed to check for already-existing links, variant grammatical forms, etc.
- I created a MsgBox routine - providing context - so that what needed to be done with potential links could easily be decided when the linking process was (re-)run.
- Links are always to Notes, but are from Notes, Archived Notes, Authors and Book & Paper Abstracts & Comments.
- 18Q1
- 17Q4
- Added %age complete to Notes Reading Lists
- Enabled deduction of Note link by omitting the Note ID from the mark-up and doing a look up on the key-word and its varients.
- 17Q3
- Added Colour-coded "quality markers" to Notes: included them in lists and jump-tables
- 16Q1
- Allowed Note- & Footnote-links from multiple words111. This is such a useful feature!
- Completed converting “static” pages to Notes.
- Enabled direct links to Notes from within Footnotes.
- Stopped (I hope!) “private” Notes showing in Paper Abstracts.
- 15Q4
- Converted the “static” pages in my website to Note format (so they can change and retain history).
- Fixed problem of images not appearing in "Printed" Notes (wrong directory level) unless the images are repeated
- 15Q3
- Tidied up the archived Notes.
- 14Q4
- Made improvements to the regeneration and referencing of archived Notes.
- 14Q3
- Created a progress-tracker112 for my Thesis.
- Improved error-recovery where encoding fails for Notes links.
- Started to split Paper Abstracts from Write-ups.
- 14Q2
- Wrote routines to import old (but missing) archived Notes into the database so they can be (and have been) re-exported in the current format.
- 14Q1
- Made a lot of improvements to my Site generators, in particular the pesky bug that stopped the links to archived Notes working correctly.
- 13Q3
- "Single Image" development for Notes
- Allowed "Authors Works Lists" to be linked-to from Notes
- Created Archived Notes Re-Export facility
- Simplified / Re-engineered Notes processing
- 13Q2
- Improved "Concatenated Notes" production
- 10Q3
- Add “Date Created” to Note_Links (for audit purposes; I’m still not convinced that links disappear when they should).
- Investigate Note_Links: needs versioning, and section of calling Note
- Wrote routines to automatically generate the “Printable Notes” footer – and to create the corresponding web pages – from the control table. Very useful.
- 09Q4
- Restricted the size of the “previous versions” tables for Notes to 12.
- 09Q3
- Enabled automatic immediate promotion to Live Website of selected Temp Notes.
- Enabled Concatenated Notes (by Group). This is only allowed for “Identity” Notes at the moment, which is what I’d wanted it for. It allows all Notes for a Group to be printed off in Title sequence, for ease of review.
- Ensured that the Notes Reading-list is generated from all 3 sub-topics.
- Functionally enhanced the Notes-generation processing options; to make niftier production possible. In particular, where individual Notes are selected, I made the re-generation of the cross-references optional (so that if only textual variation is being iterated, the iterations are quicker).
- Improved the performance of “All Notes” regeneration. The problem was with the generation of the generic reading lists based on Note Title. These only apply to the “Identity” Notes, and it was taking a long time to determine there was no list to produce.
- 09Q2
- Added a table of Notes history at the end of each Note (ie. not just previous version) - to save tedious chaining
- 09Q1
- Added reading-list (of works cited) at the end of the Note, as well as in the Printed Note
- Created optional "don't print" feature for specific footnotes.
- Fixed the problem of the overpopulation of the Notes directory
- Option for Notes-Regen:
- Changed-only
- Changed, and all impacted by Change
- All
- 08Q1
- Allowed regeneration of Temp Notes only
- 07Q4
- Added referenced Books / Papers as footnote jump table to Notes
- Added referenced reading lists to printable version of Notes
- Improve "Temp" note processing. Suppress "_T" suffix (just don't copy to Transfer)
- When a Note is archived, ensured a printable note is also archived113 (if this Note has a print)
- 07Q3
- Created "respondent notes" - database identifier & red indicator + text
- Created114 a "quick lookup" table of responses to a note and its sub-notes (recursive function on Notes_Links)
- Enabled generation of printable version of Notes
- Made Note-change-history available on-line (added status + publishable flags)
- 07Q2
- Added "footnote" pages115 for Carthusians & King's
- Added "outgoing links" summary footer to Notes pages
- Allowed two sorts of Note - document-specific ("Write-up") and general
- Papers
- 21Q2
- 20Q3
- 20Q2
- Determined why the monthly regeneration process for Paper Abstracts had ballooned from just over 6 hours in January 2020 to over 17 hours in April 2020. Reduced to just under 4 hours as a result of materialising and indexing the view (now table) Paper_Citings_List_New. Further improvements required.
- Determined why the monthly regeneration process for Paper Summaries had ballooned from just over 3.5 hours in January 2020 to around 11.5 hours in April 2020. Fixed by materialising and indexing the view (now table) Paper_Citings_List_New, so now takes 6 minutes!
- 20Q1
- Make use of Comments for large Abstracts seamless by appending Comments (if > 5k) to Abstracts.
- 18Q2
- Papers associated with broken links:-
- Created an area to hold pdfs, and automated links thereto.
- Created a pop-up to say I'd remove such pdfs if requested by their authors.
- Consequently, haven't password protected the area yet.
- 18Q1
- Correction of narratives on Note time-recording pseudo-papers
- Photos
- 19Q1
- Created Photo-logging usage via Images table.
- Created Timeline software
- Timeline software: Added photos of Coxes Farm
- 18Q3
- 13Q2
- Reduced the size of photos (to make the pages quicker to load).
- Process
- 20Q3
- Determined why Recalculation (cmdRecalculate_Click) took so long: it varied depending on how many Notes were regenerated, but seemed to be 17.5 minutes even if there are none. I think it was down to a problem with nested Functors leading to Notes growing inordinately: in particular the development logs. Now takes around 5 minutes.
- 20Q2
- Full Website Regeneration took 60 hours in April 2020. This has now been substantially fixed, as of end June 2020, in that regeneration now takes just under 11 hours. While further improvements are possible, the best use of time is to avoid the need for regular regeneration altogether by completing my Cross-referencing project.
- Corrective work was split out into the items requiring improvement, must of which have now been implemented.
- History and analysis:-
- This 'balooning' was on my new laptop which has solid state disks - it was taking 36 hours on my old laptop, then halved when I got the replacement, but then nearly doubled!
- I investigated why this was so, and improved performance mainly by putting in a trace (using a timer - GetTickCount - found on the web that allows logging of elapsed time in milliseconds, using StartTimer and EndTimer) and finding the 'pinch points'
- I suspected that it was a new release of MS Access that's caused a problem; maybe a lost index or something similar. I couldn't find any evidence of this, but a mumber of queries were taking a second or so to execute, which small times multiplied up drastically when the number of pages on my site - over 100k - are taken into account. Quite why a cliff-edge had been reached, I know not!
- Three areas had been particularly impacted: Authors, Paper Abstracts & Paper Summaries, which have all trippled or worse. These are covered as seperate developments under 'Authors' and 'Papers'.
- Book Summaries and Book-Paper Abstracts had also at least doubled, but this has had less of an impact as they took much less time in the first place.
- Other regeneration processes - in particular 'Notes' - didn't appear to have been impacted.
- Wrote Check_Database_Size to check that the size of the database isn't at risk of breaching the 2GB limit. Required because of the need to re-materialise various views that have been instituted to speed up the full website regeneration routines, and which need to be run each time cmdRecalculate_Click is run, which is often.
- 19Q4
- Full Website Regeneration failed in Notes Regen because the first Private Note (133117) couldn't be copied from the Transfer directory as the private note hadn't been created there by then.
- It should have been, from looking at the code.
- Also, the private note itself (1163118) is not regenerated until later.
- In the start-December 2019 run, no such problem occurred. Maybe the problem was caused by copied between laptops. So, I've flagged it as complete for now, pending a recurrence.
- Fix Functors to enable previous quarter's Status Reports after Time period roll-over
- 18Q4
- 18Q2
- Re-engineered Website Maintenance Dashboard120 to show extra topical links (and show them at the top of the page).
- 18Q1
- Improved efficiency of daily changes regeneration (removed msgboxes)
- Wrote a routine to update a set of .htm files, replacing text in them all
- 15Q4
- Created a consolidated processor to enable overnight regeneration of the whole website.
- 14Q1
- Created a “Website Regeneration Dashboard” to show when the various elements of my website have been refreshed, and how long the routines took to run. Links to this appear in the footer to each webpage.
- Improved the regeneration routines for the various classes of webpage, and got them to record statistics in support of the “Website Regeneration Dashboard”.
- 09Q4
- Set up my database system to roll forward to the next academic year, and generate web-pages appropriately.
- 09Q3
- Created a “performance stats” database and routines to write to it – though I only implemented them for the Notes generator.
- 07Q2
- Improved ease of FTP process by use of Transfer Directory
- Search
- 17Q2
- Wrote routines to create a cut-down Website excluding pages with irrelevant or duplicated information. Used this to create the ZoomSearch facility.
- 07Q4
- Created search facility using ZoomSearch
- Spider
- 20Q4
- System Resources Exceeded - "Run Time Error 3035" - re-emerged in Full_Link_Same_Directory_Gen. Last month I had replaced query Full_Link_Same_Directory_Updt with identical SQL, which had worked OK. This time I removed the "WHERE" statements and put them in the code itself. Seems fine.
- 20Q3
- System Resources Exceeded - "Run Time Error 3035"; query Full_Link_Same_Directory_Updt in Full_Link_Same_Directory_Gen. Fixed by using SQL rather than a query.
- The size of the main database was bloating to over 1.5Gb during the spider run, so was approaching the 2Gb limit.
- Used Check_Database_Size, with a parameter, to monitor the size of the main database and added a message similar to those reporting the compact / repair of the Slave database each time a status message is printed.
- Based on the diagnostics produced, it was found that the bloating occured at two stages:-
→ In Spider_Scurry: during the creation of Raw_Links_Temp_Temp.
→ In Spider_Copy: during the creation of Raw_Links_Temp.
- Fixes:-
→ Put check in Check_Database_Size to STOP if the database is over 1.5 Gb (parameterised via Max_Database_Size).
→ Moved Raw_Links_Temp_Temp to the Slave database.
- 18Q4
- Determined why Full_Link_Up_Levels_Gen takes so long - approximately 4 hours. Re-architected to reduce database reads.
- 16Q4
- Added “.shtml” and “.shtm” to the list of web-suffices.
- 15Q4
- Finally cracked the Spider for external links, and used it to check the 3,000-odd links from my website. Some tidying up outstanding.
- 14Q4
- Fixed the bug wherein the Spider ran out of system resources on large runs: now compacts & repairs the slave database as it does along (every 200k links; parameterised).
- 14Q3
- Continued working on my Spider and started to use it to clean up my website by diagnosing broken links.
- 13Q2
- 10Q2
- Wrote my own Web Spider.
- It’s a little bit of a cheat, in that it doesn’t actually invoke the web-links, but crawls through my directory structure selecting all the .htm files, and rummages through the HTML looking for links which it then copies to a table.
- It also maintains a table of the directory structure and files therein, and then works out the real address from the relative addressing.
- It then allows cross-checking between the links and the files.
- Links external to the site have to be investigated manually (at least I have a list of all the links) – unless I can work out how to use the list to power a browser to check.
- There are about 333k links in 55k files – the full run took about 8 hours.
- It now runs incrementally based on file last-updated dates.
- Status
- 20Q4
- 20Q3
- Added Weekly Project Plans to Priority Task List
- Created web-page (using Functor_21) showing oboe practice hours by work played
- Quarterly Project Reports: Corrected Functor_08. The Project Planned YTD % kept having to be bodged!
- 19Q2
- Corrected Quarterly Reporting for uneven Plan (Holidays)
- 18Q4
- 18Q3
- 18Q2
- Added extra items to Functors to facilitate Quarterly Reporting:-
- Automatically calculated QTD & YTD time statistics in the quarterly reports.
- Provided a YTD task list for Status Reports for projects "in abeyance" to make reporting easier.
- Modified the statistics - to be just YTD - for YTD Status reports.
- Created table for - by Project - "Plans for the near future" tasks, and a functor to add them to the status report.
- Took "items competed" and "plans for the near future from the development log for the Web-tools status report.
- Created another functor to add these lists to the Summary report.
- 18Q1
- Created web-page generator for Development items (ie. Created the Web-Project "To Do" List and "Outstanding Developments" directly from this list).
- 13Q3
- Re-engineered Monthly Reporting on Change of Year
- 13Q2
- Improved "This Month's Summary" Reporting
- Improved Quarterly Reporting
- 13Q1
- Re-engineered Monthly Reporting Tables & improved Monthly Reporting
- Technology
- 20Q4
- Consolidated Site-Hits:
- Reviewed Webalizer Methodology & Consolidated Stats
- Aggregated Robot URLs
- 19Q1
- Investigated the Hit Stats provided by Webalizer and produced web-pages to show the most active pages.
- 13Q3
- Converted Tables to CSS3 standard
- Used CSS in all manually-crafted and automatically-generated pages
- 13Q1
- Timesheets
- 17Q4
- Separated Time Recording for Aeon
- 15Q1
- Segregated time-recording so I can easily see how much time is spent developing and supporting my own website, and how much is spent on those of other organisations.
- 13Q3
- Re-engineered Time-Recording, so that actuals are held against Papers (and Books) in a separate table, rather than as columns by year in the Papers / Books tables themselves. Retained the grand totals only.
- 08Q4
- Updated Book totals from totals of the Papers contained therein
- WebRefs
- 21Q1
- Webrefs_Update failed because IE looped with Aeon / Psyche pages. Performed manual checking for a while, but then Aeon seem to have fixed their pages so the problem seems to have "gone away". It doesn't occur for other pages.
- 20Q3
- Improved WebRefs checker (Webrefs_Update) further to check for Error 403 "Forbidden". This involved finding way of checking pdfs where the returned page is in fact HTML or XML (see DevLog Ref 379).
- Reformated WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: The 'As Above" lines waste space. Consolidate onto single second line.
- Reformated WebLinks_Tester_Brief, WebLinks_Tester.htm, WebLinks_Tester_Map.htm, WebLinks_Tester_Full.htm & WebLinks_Tester_Full_Map.htm: Allowed more space for 'link returned', 'issue' and 'display text' and added Explanation column
- 20Q2
- Webrefs_Update: Improved the processing of this sub which controls IE to check the URLs corresponding to WeRefs are still valid.
- Improved the recovery processing after IE fails or becomes detached. Now seems to work perfectly!
- Improved processing for 404 (page not found) errors - detect them where URL returned differs from that requested (so the error is correctly categorised) and also where it is the same (so the error is detected) - by using the GetElementsByTagName method to look through Title and H1 elemnts (where they exist).
- Note that there is still an issue for pdfs where the above methods don't work; though the page returned is an HTML or XML page, the changed URL still claims to be a pdf, so I can't check in case it's a real pdf and the process fails.
- Amend WebRef 'Name' link to #Off-Page_Link_WxxxW style. Also amend Quarterly and Annual task lists to use this reference.
- Amended WebRef 'Name' links to #Off-Page_Link_WxxxW style so referencing from other pages is possible (useful for Aeon)
- As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 184 items. res://ieframe.dll/ in Returned_URL.
- WebLinks_Tester: If the URLs of WebRefs are changed between Spider runs, the URL mapping fails, and the WebRefs appear at the end of the last page, wrongly categorised. Fixed to ensure they appear on their own page, added to the jump table as 'WebRef Missing' and added an explanation.
- WebLinks_Tester_Full & WebLinks_Tester_Full_Map: Reformated Jump Table to 30 columns and multiple rows.
- 20Q1
- As revealed by Spider: WebLinks_Tester_4.htm. Refs failing. 32 items. res://ieframe.dll/ in Returned_URL.
- As revealed by Spider: WebLinks_Tester_Brief_12.htm (etc). Refs failing. 41 items. res://ieframe.dll/ in Returned_URL.
- As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 51 items outstanding. res://ieframe.dll/ in Returned_URL.
- WebRefs checker failing to detect broken links for Aeon - if a link fails, I get the message 'The page you were looking for could not be found (404)', but since the URL doesn't change, I don't spot it. This may happen for other sites as well. Fixed for Aeon - watch out for others.
- 19Q3
- As revealed by Spider: Hits_Pages_Total_Crosstab_Descending.htm.
- 18Q4
- Added Display text to WebRefs for Links to Stanford, Wikipedia & Aeon
- 18Q2
- 18Q1
- Determined & reported usage directly using embedded +WnnnW+ encoding rather than via the Spider. Reconciled the two methods.
- Documentation & Bug-fixes: Phase 1
- Completed "as is" functional documentation so I could see what's going on.
- Re-engineered so it made sense and ran efficiently without bugs
- Fixed the obvious data errors that confused the statistics - ie. where the returned URL differs from that requested and https issues.
- Wrote / improved diagnostic reports (web pages)
- 16Q1
- Some tidying up of the 3,000-odd external links from my website.
- 13Q2
- Some pages render oddly in Chrome (and maybe other browsers) – investigated130.
- 10Q3
- Improved the functionality for automatically generating external Web-links in Notes, Book Abstracts and Paper Abstracts. Corrected current data to accommodate the new code.
- 07Q2
- Made Web Refs in Comments / Abstracts into Hyperlinks
- Other Websites: Completed Items By Category:-
- Acts
- 16Q3
- Developed a website for displaying the textual and grammatical analyses and appendices of Pete’s PhD on the Acts of the Apostles. Two versions: Live (Acts: Live Site (Defunct)) and Test (Acts: Test Site).
- Bridge
- 17Q1
- Added new features (70s Club & Monthly Pairs Winners) to my Bridge mini-websites.
- Created Master Point Lists & Promotions for the ECBA website.
- 16Q3
- 16Q1
- Accommodated Alaric’s new scoring system for St. Edith’s.
- Created Ladies’ and Gentlemen’s ladders for my mini-websites.
- Reconciled my X-IMP calculations with a new feature provided by Bridgewebs.
- 15Q4
- Segregated the Bridge website generators from those for my own website. They now live in separate databases and have their own documenters.
- 15Q3
- Created a new website (using Bridgewebs) for the First Class Bridge Academy (First Class Bridge).
- Ported my “handicap / improvers” routines from Hutton to “Mini Websites”.
- 15Q2
- Development of the “Eve Goblets” (victor ludorum) software for the ECBA, and roll-over to the new season.
- 15Q1
- Further improvements to my Butler-IMPs and Cross-IMPs calculation routines.
- Started a project to provide small websites for bridge clubs without sites: Essex Bridge Results.
- Took on the ECBA Website (Essex Contract Bridge Association), at the cost of considerable effort by way of reformatting & tidy-up.
- 14Q4
- Improvements to my Butler-IMPs and Cross-IMPs calculation routines – different formats, including the Jeff Smith format, and extension to Butler-scored Pairs events. This work has been taken up by the ECBA selectors for county-team selection.
- 14Q3
- Created a suite of programs to extract data and calculate Butler-IMPs and Cross-IMPs from the travellers on Bridgewebs Teams events, and output ranking-lists to the Web.
- 13Q2
- Merged131 Philosophy & Bridge Databases
- Enigma
- 18Q4
- Added 'EE' shortcut icon on all Webpage browser tabs
- 15Q3
- Created a small website for a music group Julie and I attend – the Enigma Ensemble (Defunct).
- Sophie
- 19Q3
- 18Q4
- Added 'SB' shortcut icon on all Webpage browser tabs
- 15Q3
In-Page Footnotes
Footnote 6:
- There have been continual changes and bug-fixes that are not worth reporting.
- A quarterly release - or the equivalent - of small changes is to be understood passim.
- There were gaps in the development log that have been filled in from old reports. I've ignored developments for now-defunct sites.
- However, for my own site I've erred in the direction of plenitude to remind me of what went on. I may prune the list in due course.
Footnotes 37, 77:
- This situation occurs in these reports, and elsewhere.
Footnotes 38, 78: Footnotes 47, 104:
- An eccentric use of this term.
- By it I mean the extraction of data from a database, formatting it, and inserting the resulting text in line into a web-page.
Footnotes 54, 111: Footnotes 55, 76:
- I subsequently found a bug whereby it was backing up the entire code-base each time!
Footnotes 56, 132:
- Sophie is an old friend and supervisor from Birkbeck.
Footnotes 58, 72:
- Naturally, I can now say precisely what I mean, by using a plus-sign, as it no longer causes the process to fail!
Footnotes 59, 130:
- Fixed – it was very long blank-free URLs in the Source column.
- I now call my “WebRef” function to convert these to hyperlinks.
Footnotes 60, 131:
- This was a mistaken idea, and was subsequently reversed!
Footnotes 61, 128:
- Rather irritatingly, when I checked my site in all browsers, I found that it appeared a right mess in some of them (I forget the details).
- It all stemmed from the DocType parameter, which I had to revert.
Footnotes 62, 113:
- I must have unwound this one - presumably it didn't work - and need to re-do it.
Footnotes 63, 114:
- Did I - or was this an intention?
- The whole "respondent" process didn't work out well as it fragments the discussion, so this might be important.
Footnotes 64, 73:
- This must have been unwound if it was ever implemented.
Footnotes 65, 74:
- "BB" = "Blackberry" (for when I had one!)
- Response times - eg. in bookshops - used to be awful. This "binary chop" approach is no longer needed.
Footnotes 66, 67, 69, 79, 80, 115: Footnotes 68, 82:
- Presumably this was the ancester of this page before it was converted to a Note?
Live Version of this Archived Note
Table of 12 Earlier Versions of this Note (of 20)
Table of the 8 Later Versions of this Note
Summary of Notes Links from this Page
To access information, click on one of the links in the table above (if any).
Summary of Note Links to this Page
| Status: Web-Tools (2021 - June) |
Website - Outstanding Developments (2021 - June) |
Website - Outstanding Developments (2021 - September) |
Website - Progress to Date (2021 - June), 2, 3, 4, 5, 6, 7, 8 |
Website Generator Documentation - Functors, 2, 3, 4, 5 |
To access information, click on one of the links in the table above (if any).
Text Colour Conventions
- Blue: Text by me; © Theo Todman, 2024