Theo Todman's Web Page
For Text Colour-conventions (at end of page): Click HereStatus Reports - Status: Web-Tools (2018 - September)
Rationale for this ProjectNote last updated: 10/10/2018 16:43:41
- This Project was alluded to briefly in a footnote on research methodology in my original Research Proposal under the head Research - Internet Technology. When last at Birkbeck, I wrote a more extensive paper defending the Project and describing its rationale. Now that my PhD is in suspense, I have decided to take this Project further. There’s a lot to do: still quite a few items on the “wish list”. It is fairly critical as an enabler for my research, so I need to get a move on as I want it all out of the way before I re-start4 formal research.
- For documentation on my website (currently password protected) follow the links below:-
- Functional5 Documentation.
- Technical7 Documentation.
- Other Websites
- Sometime around 2005, I created a website for Dr. Sophie Botros (Sophie Botros: Live Site (http://www.sophiebotros.com/)), one of my supervisors at Birkbeck, but we then lost touch and it got maintained (very badly) by some desktop support outfit. In 2Q15 I took it back on again and spruced it up a bit, and maintain it periodically, though it still requires more work.
- I created and / or ran a multitude of bridge websites, but as of January 2018 I have either handed them over or mothballed them9:-
- I established the Hutton Bridge Club (http://www.bridgewebs.com/hutton.html) Website in 4Q11 using the standard Bridgewebs service, but with a couple of competitions using my own routines. This was handed over in 3Q15.
- In 1Q15, I took over the support and development of the Essex Contract Bridge Association (http://www.bridgewebs.com/essex/) (ECBA) website, which also uses Bridgewebs, but is very much larger. I wrote a lot of code10 to make this job less tedious. The site was handed over in 4Q17.
- For several years, I collected data11 on bridge activity in the Billericay / Brentwood area (initially needed for a project to set up a new consolidated club) – by “scraping” data off web pages, consolidating it into a database and modelling it in various ways.
- I used this data to generate websites with a multitude of ladders for small clubs (Essex Bridge Results (http://www.theotodman.com/Bridge/index.htm)). These are now mothballed.
- I created and maintained a new website for the First Class Bridge (http://www.bridgewebs.com/firstclassbridge/) Academy, giving it “small clubs” ladders as these were easy to maintain with little intervention.
- In 3Q16 I revived the Mountnessing Bridge Club Archive (http://mountnessing.theotodman.com/index.shtml) website.
- I’ve created and continue to maintain a small website for a music group Julie and I attend – the Enigma Ensemble (http://www.enigmaensemble.co.uk/).
- I created a website for displaying the textual and grammatical analyses and appendices of Pete’s PhD on the Acts of the Apostles. It exists in two versions: Acts: Live Site (http://www.acts-research.website/) and Acts: Test Site (http://www.theotodman.com/Petes_PhD/index.htm).
- I did consider returning to work part-time as an MS Access/Excel developer, with a spin-off into website generation, but have done nothing about it so far.
Summary of Progress during July - September 2018
Website (Total Hours = 139)
- I spent 141 hours in 3Q18 on this Project, or related work (643 hours YTD, where for "YTD" - Year to Date - I mean the (academic) year that commenced in October 2017). That's 90% of the planned effort (93% YTD). Overall, 24% of my Project effort in the Quarter was directed towards this project (making 27% YTD) - as against 31% planned (27% YTD).
- I’d been on course to exceed budget, but decisions on other projects – in particular Music and Chess – got in the way towards the end of the quarter, so I only achieved 90% of plan in the end.
- However, I did manage to spend almost 140 hours developing my own site and did a lot of useful work.
- Completed items included:-
- Added 'TT' shortcut icon on all Webpage browser tabs
- Authors Narrative: enable footnotes.
- As revealed by Spider: 'Extra' Links to Notes from Citations List have the wrong directory (clumped in 1,000's like Papers rather than 100's). 7 items. Problem with AddCitations_List, so not just an issue for Authors.
- As revealed by Spider: BookCitings. 405 items. Fixed problem with CreateBookCitingsWebPages not checking for change of Book during 'Repeats' processing.
- Paper & Book Abstracts - Make 'Disclaimer' more prominent & show 'Paper Statistics' rather than 'Paper Summaries'
- Reformated + Fixed Hand-dealing pages to CSS standards
- Created Functor using Form_Documentation_Links to automatically add control-links to documentation Notes.
- Fixed Bug where a variable has the same name as a procedure, and is thereby confused with one. Produced a warning query (Variable_Code_Name_Clashes) for manual correction.
- Enabled the re-use of Footnotes in the same Note12. They appeared multiple times, now just once. Fixed for:-
All of which use Notes_Text_Format and the Note_Footnotes table. Also,
- Notes (OutputNotesWebPage)
- Printable Notes (NoteForPrinting), and
- Archived Notes (OutputNotesWebPage_Archived)
- Book & Paper Abstracts & Comments used to use an array in Reference_Footnotes, but changed to use a new table13 Abstract_Footnotes.
- Created auto-regen of all Note Link (Note_nnn_Link) Pages
- Automatic Key-word Hyperlinking to Notes Project: Phase 2
- Added links based on Book & Paper Titles
- Added ***Read***-marker to Lists
- Allowed links to be ignored (rather than suppressed) during approval process
- Skip links to Status Reports, etc.
- As revealed by Spider: Notes_List_Control links fail for Supervisions. 58 items.
- As revealed by Spider: Links to Notes_xxx_Links, from Abstracts (94 items), Notes (25 … corrected manually), NotesPrint (1916). Fixed by +LL+ functor implementation.
- As revealed by Spider: Bridge Status Report Achived Notes - sundry links failing. Live Note OK. 70 items.
- As revealed by Spider: Languages Status Report Achived Notes - links to Dissertation failing. Live Note OK. 43 items.
- As revealed by Spider: Religion Status Report Achived Notes - sundry links failing. Live Note OK. 59 items.
- Created new '+LL+' Functor for links to Note_Link pages; fixed associated Notes.
- Blog: Created Functor to generate Blog from Blog.
- Fix Bug: photos in Abstracts with embedded Notes - eg. "Grossman (Lev), Kurzweil (Ray) - 2045: The Year Man Becomes Immortal" - not displaying
- As revealed by Spider: Fixed ImageRef to work for both embedded ('SRC') & linked images on the same page - eg. Carthusians. 12 items.
- Status: Summary - Actual versus Plan between quarters says "previous quarter"; should be "current quarter"
- Full details for 18Q3 are given below:-
- Website - Bridge Development
- Website - Development (Total Hours = 97)
- Author Name Format options (Surname, First-Name) (0.5 hours)
- Automatic Key-word Hyperlinking to Notes Project: Add links based on Book & Paper Titles (1.5 hours)
- Automatic Key-word Hyperlinking to Notes Project: Add Read-marker to Lists (1 hour)
- Automatic Key-word Hyperlinking to Notes Project: Allow links to be ignored (1 hour)
- Automatic Key-word Hyperlinking to Notes Project: Automate full regeneration (6.25 hours)
- Automatic Key-word Hyperlinking to Notes Project: Bug for Suppressed Links in Book-Paper Abstracts (1 hour)
- Automatic Key-word Hyperlinking to Notes Project: Refinement of 'outstanding' counts (0.5 hours)
- Automatic Key-word Hyperlinking to Notes Project: Refinement of title re Exclusions (1 hour)
- Automatic Key-word Hyperlinking to Notes Project: Skip links to Status Reports, etc. (0.5 hours)
- Technology: Added TT shortcut icon (1 hour)
- Technology: Added 'TT' shortcut icon (0.25 hours)
- Technology: Look into Sistrix Smart (2 hours)
- Website - Generator - Allow Footnotes for Author Abstracts (1 hour)
- Website - Generator - Allowed links to both embedded ('SRC') & linked Images on the same page (1.75 hours)
- Website - Generator - BookCitings Links failing (2.75 hours)
- Website - Generator - Bridge - Reformat + Fix Hand-dealing pages to CSS standards (4.75 hours)
- Website - Generator - Create Functor for adding code links to Website Functional Documentation (2 hours)
- Website - Generator - Create Functor to generate Blog (4 hours)
- Website - Generator - Create new '+LL+' Functor for links to Note_Link pages (5 hours)
- Website - Generator - Documenter Bug-fixes: Confusion of Variables with Procedure Names (0.75 hours)
- Website - Generator - General: Tidy up Site - delete un-updated pages (3.5 hours)
- Website - Generator - Improve efficiency of Quarterly + YTD Task List Generation (0.75 hours)
- Website - Generator - Links from Notes_List_Control failing for Supervisions (0.5 hours)
- Website - Generator - Links from old Status Reports failing (4.25 hours)
- Website - Generator - Links to Images in embedded Notes in Abstracts failing (0.75 hours)
- Website - Generator - Maintain consolidated Development Log (3.75 hours)
- Website - Generator - Paper & Book Abstracts - Clarify Disclaimer & 'Summaries' (2 hours)
- Website - Generator - Problem with AddCitations_List and directory for links to 'Extra' Notes (0.5 hours)
- Website - Generator - Re-Document Front Screen (5 hours)
- Website - Generator - Remove duplicate footnotes (7.5 hours)
- Website - Generator - Status: Summary - Actual versus Plan; Say 'Current' Quarter if current quarter (0.75 hours)
- Website - Generator - Stop Test Note generating Book & Paper Links (0.5 hours)
- Website - Generator - Web Spider - Check out & correct missing links (8.25 hours)
- Website - Generator - Web Spider - Documentation & Diagnostics (9 hours)
- Website - Generator - Web Spider - Investigate Run-time (0.25 hours)
- Website - Generator - WebRefs - Add Test Page Regen to Maintenance Dashboard (2.25 hours)
- Website - Generator - WebRefs - Check out Quarterly Detail Report - Paper Links (0.75 hours)
- Website - Generator - WebRefs - Investigation & Correction of Errored URLs (7.25 hours)
- Website - Generator - WebRefs - Run periodic full check (0.25 hours)
- Website - Generator - WebRefs - Spurious entries generated by Spider (0.75 hours)
→ See "Software Development - Website - Development"
- Website - Education (Total Hours = 16.25)
- Website - Infrastructure (Total Hours = 6.75)
- Check out email@example.com email account (0.25 hours)
- iCloud Photos directory problem after Windows 10 release - turned off iCloud (1.25 hours)
- Investigating repairing unreadable DVDs (0.25 hours)
- Julie's Laptop Support - Including Windows 10 Bugs & Upgrades (0.25 hours)
- Microsoft Windows 10 / MS Office - Releases & Bugs (0.5 hours)
- Microsoft Windows 10 / MS Office - Releases, Bugs & Periodic Re-boots (2.25 hours)
- PC Backups / OneDrive (1 hour)
- Printer Toner + Paper (0.25 hours)
- Review Webalizer stats & methodology documentation (0.25 hours)
- Website - Namesco Email problems (0.5 hours)
→ See "Software Development - Website - Development"
- Website - Maintenance (Total Hours = 16.75)
- 18Q2 Status Reports (4.25 hours)
- Website - Generator - General: Tidy up Site - delete un-updated pages (2.25 hours)
- Website - Generator - WebRefs - Manual URL Checks (0.5 hours)
- Website - Periodic Full Regeneration (6.5 hours)
- Website - Run Web Spider (2.25 hours)
- Website - ZoomSearch database refresh (1 hour)
→ See "Software Development - Website - Development"
Website Others (Total Hours = 1.75)
- Website Others - Bernie's Website Development
- Website Others - Enigma Ensemble
- Enigma Ensemble Website - Creation, Admin & Maintenance (1 hour)
Plans for the Near Future
I’ve retained the planned weekly effort on this project at 12 hours. While there’s still a lot on the “priority list”, my intention for the coming academic year is to focus on updating my technical competence. The Plan below is taken automatically from the Priority 1 items on my Development Log, as published in my Outstanding Developments Report.
- Own Website: Priority 1 Items By Category:-
- As revealed by Spider: Old links in unregenerated pages. 8 Items.
- Complete XRef-re-engineering project:-
- Ensure all links and link-pages use the new XRef table, and pension off the old tables.
- Check all link-types still work and fix any errors.
- Complete the auto-triggering of regeneration of “associated” link pages.
- Fix update bug in Convert_Webrefs.
- Fix Bug whereby PaperSummary pages seem to have “Works-” and “Books/Papers-” Citings that refer to the same link-pages.
- Review effectiveness of hyperlinking method in the light of PhD and Philosophy of Religion experience.
- Where possible, use ID rather than NAME for in-page hyperlinks
- As revealed by Spider: Philosophers_Index_List_OA.htm. Author Names Scrambled. 133 items.
- As revealed by Spider: Sundry broken Links where author name corrected. 93 items.
- Review architecture to improve performance; Need to document first
- Investigate whether multiple Subject/Topic/Subtopic usage leads anywhere (ie. are just the first (of 3) actually used). Fix anything amiss.
- Provide Functional Documentation for Website Generator (using Notes)
- Investigate Bootstrap.
- Plan what to do with "Sitepoint (Learnable) - Sitepoint Learnable Web Development Courses" and the eBooks in my possession.
- Read "PC Pro - Computing in the Real World".
- Read "Barnes (Russell), Ed. - Web Designer".
- Re-start "Harvard University - CS50 Introduction to Computer Science". Maybe just "audit".
- Understand Bitcoin & Blockchain better
- Re-install iCloud: solve 'The upload folder for iCloud Photos is missing' problem
- Add "Note Alternates" to Note pages.
- Allow the option to concatenate Notes in the Printed version (ie. linearly embed them essay-style), rather than treating the hyperlinks as footnotes – but still keep the hyperlink & cross-referencing in place.
- For use as "disclaimers" - eg. for "Plug Notes".
- For Thesis / essays: the difficulty here is the need for linking passages to make the text run smoothly.
- As revealed by Spider: #Reference links: fail for Printable Notes. 24 items.
- As revealed by Spider: Consciousness Status Report Achived Notes - sundry links failing. Live Note OK. 46 items. 24 items outstanding.
- As revealed by Spider: Blog Achived Notes - sundry links to Printable Notes. Live Note OK. 110 items. 100 items outstanding.
- As revealed by Spider: Links to sundry 'static' pages failing. 26 items. 20 items outstanding.
- As revealed by Spider: Old links from Archive Notes to (non-existent) Archived Printable Notes. 42 items.
- As revealed by Spider: Status Reports - Achived Notes - sundry links failing. Live Notes fixed.
- Website: Maintenance Dashboard. 14 items.
- Personal Identity Status Report. 11 items.
- Summary Status Report. 4 items.
- Actual Detail Summary Report. 6 items.
- Chess Status Report. 12 items.
- As revealed by Spider: Sundry links fail for Supervisions. 31 items.
- Created auto-regen of all Note Link (Note_nnn_Link) Pages
- Investigate Note_Links: Section references seem to be incorrect
- Printable Notes: fix the bug whereby the “private” flag is round the wrong way.
- Suppress the publication of the Printable versions of Temp Notes
- Add photos of Tom - Create Timeline software
- Develop software & procedure to make adding more content to the photos pages easier to undertake.
- As revealed by Spider: Sundry uncategorised. Refs failing. 24 items.
- Determine why Recalculation & Changed Book/Papers produce unneeded regeneration.
- Analyse the results of the data collection exercise and design a plan of campaign to fix broken Internal links and prevent recurrence.
- Correct the code so the problems discovered by the Spider don’t recur.
- Delete 'orphan pages' that are never linked to, ie. Use the Spider to prune redundant pages27 automatically where possible.
- Fix the historical data where errors are uncovered by the Spider. An easier task now the site has a full-regen function.
- Determine why Full_Link_Up_Levels_Gen takes so long - approximately 3 hours. May need to re-architect.
- The Spider was generating WebRefs. Procedurally, this ought not to have been possible.
I've re-opened the case!
- The major problem turned out to be because unprocessed28 URLs got added to the end of the last WebLinks_Tester_Brief page, which then got Spidered. I've stopped this happening, so hopefully the problem will not recur. The fix was made in 18Q2.
- However, 4 other creations appeared - dated 18/05/18 - from the run of 10/07/18. The creation date was from the previous spider run, but the IDs show that they were produced in the latest run.
- Look into Sistrix Smart29. Errors and warnings itemised are:-
- Duplicate content: seems to be theotodman.com and Link - Defunct
- Title Tags: Empty, too long, identical
- Page Not Found
- Filesize in excess of 1Mb
- Meta-Description: Empty
- Few words on Page
- H1: Not used, used multiple times per page, identical across pages
- Pictures: Alt attribute missing
- As revealed by Spider: WebLinks_Tester_Full_Map.htm (etc). Refs failing. 116 items.
- As revealed by Spider: WebLinks_Tester_Map_3.htm (etc). Refs failing. 21 items.
- Documentation & Bug-fixes: Phase 2
- Re-document the procedures in the light of recent changes.
- Resolve issues generated / revealed by the spider.
- Investigate - and fix where possible - broken links.
- Other Websites: Priority 1 Items By Category:-
- Create "Small Sites" database for Alaric (if chased)
- Termly updates to the Enigma Ensemble (http://www.enigmaensemble.co.uk/) Website.
- Periodic updates to Sophie Botros: Live Site (http://www.sophiebotros.com/)
Summary of Progress to Date
This is hived off to various separate documents, which have now been harmonising and / or consolidated:-
- Summary of Progress to Date.
- Outstanding Developments,
- Functional Documentation,
- A summary of time expended across the years developing my website33 is at "Software Development - Website - Development".
- Well, I’ve missed the boat, so will need to continue with both projects in parallel.
- This was always likely to be necessary, as new features will always arise in use. It’s a prototype methodology, after all.
- This is very tedious to produce and consequently is both incomplete and out of date.
- This is much more fun, as it’s a purely technical task.
- I’ve written a vastly-improved general-purpose technical documenter for MS Access.
- It’s a shame to abandon the “mini websites” with all their ladders, as it’s rather well done.
- However, I can’t waste time on these after I’ve abandoned bridge.
- In particular, for the ECBA “Victor Ludorum” competition.
- I cannot hand any of this code over, so the tedium will return, though not to me!
- I had agreed to share this data sometime early in 2018
- But will wait until asked again, as I doubt it’ll be of any real use to anyone.
Footnote 13: Footnote 27:
- This situation occurs in these reports, and elsewhere.
- Note that Backup_Prune_Ctrl deletes (relevant) pages that weren't regenerated in the last full site-regen, but this isn't the same thing.
- These are URLs that were used in web pages but hadn't yet been converted to the +WnnnW+ format, so appeared at the end with no WebRef ID.
- See Sistrix (https://www.sistrix.com/smart/)
- This used to be called Optimizr, see Optimizr (http://www.optimizr.com/) (which now auto-forwards to Sistrix).
- A quick look doesn’t show it to be an obvious scam, but I need to double-check.
- An unsolicited analysis of my site turned up monthly from Optimizr from January 2015 to October 2017, listing a large number of “problems” that I think I know about, but which are in the queue to address.
- It restarted in February 2018, under the Sistrix name (this seems to have been associated with Optimizr since November 2015).
- The free version of this software is restricted to 1,000 pages, which is a very small proportion of my Site, though I may be able to point it to difference base-URLs.
- But I do need to address the problems validly itemised, and a sub-set is still useful.
- As distinct from developing other peoples’ websites – time which is also recorded against this project, but not against this task.
Text Colour Conventions
- Black: Printable Text by me; © Theo Todman, 2018
- Blue: Text by me; © Theo Todman, 2018
Return to Home page
Timestamp: 12/10/2018 09:32:12. Comments to firstname.lastname@example.org.