Theo Todman's Web Page - Notes Pages

Website Documentation

Website Generator Documentation - Timelines (Photos & Blog)

(Work In Progress: output at 29/06/2025 23:54:14)

Previous VersionsNote ReferencesNote Citations

This document covers1 the following functions performed by clicking buttons on the front screen:-

  1. Update / Output Timeline Pages (cmdTimelines_Click)

To see the Code, click on the procedure name above.

Introduction
Detailed Processing – Maintenance of Static Data
Detailed Processing – Blog25
Detailed Processing – Coxes Farm Timeline31
Detailed processing – Generic Timelines
  1. Overview:
    • This function is much more sophisticated than the items immediately above, and allows the importing of photos in bulk.
    • These timelines use Functor_16 to generate the table in the webpage, using the MS Access tables Photos_Raw and Photo_Narratives.
    • The names of photo JPEGs are copied from the selected directory33 into the table Photos_Raw, classified by (currently) one of four34 photo-types, reflecting the four options above, ie:-
      1. Coxes Farm Repairs35: “CoxesFarmRepairs”.
      2. Coxes Farm Pre-Repairs36: “CoxesFarmPreRepairs”.
      3. Coxes Farm Gardens37: “CoxesFarmGardens”. For some reason 9 of these38 have Timeline = “CoxesFarmGarden” (singular).
      4. Henry39: “Henry”.
    • Optionally, the photos themselves are copied to the Photos/Notes directory on the local copy of my website (they must be copied over manually to the live site).
    • Large and small copies of the same photo are matched up – using the string immediately before the “. jpg40” to the preceding “_” (if any) – so that the small photo appears on the grid, for rapidity of loading, the large photo only being displayed on request.
    • I found a useful free utility – Mass Image Compressor – that can compress images in bulk. It works fine, and quickly.
    • The photos are segregated by month and displayed in descending date order. The date used is the “date taken” in the JPEG header41, if there is one, otherwise a manually-entered date 42 can be used. If both exist, the manually entered date takes precedence.
    • Within the month bands, a further segregation is allowed by photo-subgroup.
    • A narrative associated with each Month (or Subgroup within Month) can be entered to the Photo_Narratives table.
    • A jump-table (by Month, not sub-group) appears at the top of the page to allow easy access to, and an overview of, the timeline. The Narrative for the month appears as a pop-up.
    • An issue was display of portrait photos, which appeared as landscape in the grid, and when rotated distorted the grid unacceptably. I sorted this eventually, and flag such items – manually rotated using the standard Microsoft Photos utility – within Photos_Raw.
    • A final issue was the date the photo was taken. This can often be deduced from the JPEG header, using a modified version of a utility I found on-line (now incorporated into GetProperty). Otherwise, the date can be encoded in the file-name43, or physically typed in to the Photos_Raw table.
  2. Photo Preparation: Generic Timelines
    • New photos can optionally44 be imported into the database and then incorporated into the web pages and exported to the website. Before this, they need to be made ready – that is, extracted from the camera and placed in the right directory at the right resolution.
    • In general, as noted, I have two versions: a ‘small’ (compressed) version – which is what is initially presented on the page – and the original (full size, though this itself might have been compressed from the full-resolution photo captured by the camera) which is displayed if the ‘small’ version is clicked on.
    • There’s a separate root directory for each of the three generic timelines, as noted below.
    • Each of these directories has three sub-directories:-
      1. Originals: The latest batch.
      2. Compressed: The ‘small’ versions of the latest batch.
      3. Saved: Presumably45, a holding spot for multiple batches, while I decide what – if anything – to do with them after they have been processed. They are not really needed further, so can be deleted when the dust settles.
    • The root directory ultimately – ie. before running the import routines – contains the contents of the ‘Originals’ and ‘Compressed46’ directories.
    • My policy – to make things easier – is to import more photos than will ultimately be selected – I can see how they look on the page and then fiddle with the control tables so superfluous ones are eliminated. How are the rejected items not copied over to the live and test sites?47
  3. Photo Compression
    • The utility Mass Image Compressor 3.1: Compress Images of Directory is used.
    • A later version is available from SourceForge: Mass Image Compressor.
    • It’s amazingly quick and perfect for what I want.
    • I put the photos to be compressed in ‘C:\Theo's Files\Mass Image Compressor\Input’.
    • I then chose this as my input directory and chose ‘C:\Theo's Files\Mass Image Compressor\Output’ as the output directory.
    • Parameters are:-
      → Compression Quality: 80%
      → New Dimension: 35%
      These aren’t set in stone but seem to be OK.
    • After the conversion has been run, the output files are copied to the ‘Compressed’ sub-directory of the appropriate Timeline photos directory.
  4. The following subroutines and functions are called into play:-
    1. cmdTimelines_Click:
      • This (which is invoked for all the 9 timeline processes) does some pre-processing48 for the generic timelines:-
      • Firstly certain parameters49 are set. The importance and use of these will be explained in due course:-
        1. If i = 6
          → Photo Query = "Coxes_Farm_Repairs"
          → Photo Type = "CoxesFarmRepairs"
          → Directory_From = "C:\Theo's Files\Coxes Farm Repairs\Photos_Import_Repairs\"
          → Message = "Import more Photos for Coxes_Farm_Repairs?"
        2. If i = 7
          → Photo Query = "Coxes_Farm_Pre_Repairs"
          → Photo Type = "CoxesFarmPreRepairs"
          → Directory_From = "C:\Theo's Files\Coxes Farm Repairs\Photos_Import_PreRepairs\"
          → Message = "Import more Photos for Coxes_Farm_Pre_Repairs?"
        3. If i = 8
          → Photo Query = "Coxes_Farm_Gardens"
          → Photo Type = "CoxesFarmGardens"
          → Directory_From = "C:\Theo's Files\Coxes Farm Repairs\Photos_Import_Garden\"
          → Message = "Import more Photos for Coxes_Farm_Gardens?"
      • As can be inferred from the above, importing (more) photos from the selected directory is optional. If the answer to the above question(s) is ‘yes’, the following routines are called, in order:-
        1. Create_Timeline: with parameter set to 6 to 8 as necessary. Why is this important process run so early?50
        2. Photo_Preparation: with parameter ‘Photo Type’.
        3. Photo_Copy: with parameters Photo Query and Directory_From (set above).
      • Then, optionally, the Photo_Narratives table is updated manually to add or amend monthly titles for the timeline. The process STOPs while any updates are performed.
      • Next, optionally, we can update the Photos_Raw table (to add / amend the narratives of or include / exclude individual photos). The process STOPs while any updates are performed. On resumption, sub Photo_Preparation is run, with parameter Photo Type (set above).
      • Finally, though optionally, the timeline web page is output. CreateNotesWebPages is called after a row is inserted on an emptied Notes_To_Regen table – after the usual parameters (global variables) are set. The user is reminded that51 the status of the relevant Note has to be set to ‘Temp’ for the Functor to re-import new data. Again, this processing is repeated in the code52 for each of the three ‘generic’ options.
      • The process then terminates.
      • All the above can be performed – and refined – iteratively by re-running the process from the top.
    2. Create_Timeline:
      • This is the controlling routine. We start by decoding the parameter53:-
        1. Case 6
          → DirectoryName = "C:\Theo's Files\Coxes Farm Repairs\Photos_Import_Repairs"
          → Photo_Type = "CoxesFarmRepairs"
          → Photo_Source = "iPhone"
        2. Case 7
          → DirectoryName = "C:\Theo's Files\Coxes Farm Repairs\Photos_Import_PreRepairs"
          → Photo_Type = "CoxesFarmPreRepairs"
          → Photo_Source = "iPhone"
        3. Case 8
          → DirectoryName = "C:\Theo's Files\Coxes Farm Repairs\Photos_Import_Garden"
          → Photo_Type = "CoxesFarmGardens"
          → Photo_Source = "iPhone"
      • The process now reads through all the files in the supplied directory and checks if54 the file name is in the Photos_Raw table, adding it if it is not.
      • Before adding the record, ‘.JPG’ is converted to ‘.jpg’ in the file name.
      • Then up to three dates are extracted:-
        1. If the file name starts with ‘CF_’ it is assumed to contain a date in ‘YYMMDD’ format. This is parsed and placed in Photo_Timestamp_Manual.
        2. The file creation date is placed in File_Timestamp_Created
        3. Sub GetProperty is called to extract property 12 from the JPEG header. This is parsed and if it’s a date in the correct format it’s placed in File_Timestamp.
      • The file size is placed in File_Size.
      • After reading in all the records, a msgbox is opened showing how many files55 have been imported. When this is OKed, the sub is exited.
    3. GetProperty:
      • This is a modified version of a bit of freeware found on-line (originally in VB). It returns “Property n” from the JPEG header. I only use it with n=12, which is “date photo taken”. It is not always set.
    4. Photo_Preparation:
      • This is a fairly complex subroutine that performs a sequence of tasks on the Photos_Raw table for all rows for the requested Photo Type (this is set in the calling subroutine to the generic timeline under preparation).
      • The following three fields are set up:-
        1. Photo_Source: Initially taken from Photos_Raw, but if the file name contains ‘P101’ it is changed to ‘Olympus Digital Camera’.
        2. Photo_Popup_Title: Set up as a concatenation of:-
          → File Name
          → Photo Source
          → File Timestamp
          → Photo Narrative
        3. Link_Key: This is taken from the File name: it’s usually the text between the final ‘_’ and ‘. jpg56’. If there’s no such thing, it’s ‘None_’ concatenated with the Photo ID (though there are none of these at present).
      • The corresponding fields in Photos_Raw are then updated with these values.
      • Next, the Photos_Raw table is re-read in descending Photo_Size within Link_Key sequence, ignoring those already flagged as Excluded. Basically, by moving back and forth on this file, photos of size < 1Mb are flagged as “Medium’, otherwise ‘Full’. If there’s more than one ‘Medium’ photo for the same Link_Key, the earlier (larger) is flagged ‘Medium - Larger’.
      • We now check for inconsistent Photo-Types between different-sized images of the same photo:-
        1. Rows are added to a ‘dummy57’ table of link-keys: Photo_Raw_Excluded_Link_Keys using query Photo_Type_Goup_Counts which in turn runs off Photo_Type_Goupings. The idea is to check that the same Link Key isn’t associated with more than one Photo Type (Timeline).
        2. If there are, query Photos_Raw_MultiType_List is opened which allows Photos_Raw to be edited – basically allowing errant photos to be excluded.
        3. The process STOPs while this manual update is undertaken.
      • We now check for inconsistent "exclusions".
        1. Query Photo_Raw_Excluded_Link_KeysQ is used to add the keys of excluded photos to Photo_Raw_Excluded_Link_Keys.
        2. Then query Photos_Raw_Excluded_Inconsistencies is run off this table as a recordset. If this recordset has more than three columns, it indicates inconsistencies.
        3. in this case, the following two queries are opened58:-
          → Photos_Raw_Excluded, and
          → Photos_Raw_Excluded_Errors
        4. The process STOPs while Photos_Raw_Excluded is used to update the now easily accessible rows of table Photos_Raw.
        5. The query Photos_Raw_Excluded_Errors appears to be corrupt, having invalid field names.
      • Finally, check for where 3 or more photos for same ID are not excluded:-
        1. Query Photo_Raw_Non_Excluded_Triplets adds rows – if any – to the generic59 keys table Photo_Raw_Excluded_Link_Keys.
        2. If the table is non-empty, Photos_Raw_Excluded is opened.
        3. The process STOPs while Photos_Raw_Excluded is used to update the now easily accessible rows of table Photos_Raw.
    5. Compressed_Photos_Copy:
      • This is a routine that copies already compressed images from a selected directory and copies them to another selected directory, prefixing the file name with “Small_”. This is just to save me the bother of pasting in “Small_” so many times and making errors!
      • Files are copied from a generic60 directory C:\Theo's Files\Coxes Farm Repairs\General_Temp_Import\Compressed.
      • Currently, the ‘copy to’ directory is hard-coded and not parameterised61: it needs to be changed manually for each different timeline run. Currently set to ‘C:\Theo's Files\Coxes Farm Repairs\Photos_Import_Repairs\’. As noted above, they are copied with ‘Small_’ prefixed. Also note that this is the root, rather than the ‘Compressed’ directory for this timeline.
      • A check is made that the file doesn’t already exist in the ‘copy to’ directory. Files are not overwritten. Instead the process STOPs. I’m not sure what I’m to do62 as it continues with the next file thereafter. I suppose I could terminate the process and investigate.
    6. Photo_Copy:
      • Copies photos to the ‘C:\Theo's Files\Websites\Theo's Website\Photos\Notes\’.
      • It is called with two parameters:-
        → strPhotoQuery, and
        → Directory_From
      • These were set when the Timeline was selected from the menu.
      • Both the Full and Medium JPEGS are copied (names taken from columns 1 – Photo_FileName – and 10 – Matching_Photo_FileName). The ‘Medium’ JPEG usually has ‘Small_’ prefixed!
      • Already-existing photos are not overwritten (the processing just carries on).
    7. Photo_Copy2:
      • There are two runs63:-
        1. The first orders the photos from disk so they can be matched to those taken from the iPhone
        2. The second prefixes the date to the front (with a CF_ prefix), with the original ID to the rear
      • The driving query is Coxes_Farm_Photos_Backup:-
        1. Runs off the Backup_Site_Map table64,
        2. This query selects anything from ‘C:\Theo's Files\Photos\Coxes Farm\’, but
        3. Ignores items for:-
          → Root
          → "Coxes Farm_201307\" And <>
          → "Lych_Gate\"
          → "Survey\"
          → "Sylvia\"
          → "Coxes Farm Selective Delapidations\"
      • The process outputs to directory "C:\Theo's Files\Photos\Coxes Farm\Temp\".
      • … to be completed, maybe! The process doesn’t seem to be called from anywhere else in the system.
    8. Images_Add:
      • This Function is called by65 ImageRef to add rows concerning images – in this case photos – to the Images table.
      • There are four parameters:-
        1. Directory: this seems to be unused!
        2. Image Reference: This is the File Name66 of the image.
        3. SRC?
        4. Link?
      • The function returns the Image ID (the autonumber generated for new Images) or ‘9999’ for already-existing Images as the value of the function.
      • The Images table holds the Image ID (autonumber), the Image’s Directory and File Name the timestamp and two Y/N flags67.
      • The directory is always set to ‘C:\Theo's Files\Websites\Theo's Website\Photos\Notes\’. So, what’s the parameter for?
      • The File Name is set to the Image Ref, unless this is more than 250 characters long, in which case68 an error is presumed and ‘Dud: ‘ is prefixed to the leftmost 250 characters of the Image Ref.
      • The timestamp is always set to ‘Now’. This means that after a full website regen, all the active Image rows have timestamps set to when the regen ran. At least thereby we can see which Images are active!
      • If a new row is being added, SRC? And Link? are set from the parameters. If it’s an update, they are updated only if the parameters are set to ‘True’.
    9. Functor_16:
      • The processing of Functor_16 should be documented in my Note on Functors69, but it seems it’s just listed.
      • I’ll document it here for now, then maybe move it.
      • The Functor is called with 3 parameters: Note_ID, Note_Title, Note_Text
      • The query selection – based on Note_ID – is a bit clunky, and requires making more generic:-
        → 1278: Coxes_Farm_Repairs
        → 1282: Coxes_Farm_Pre_Repairs
        → 1283: Coxes_Farm_Gardens
      • All these queries return all the non-excluded photos from Photos_Raw for the timeline in question in reverse date order that satisfy one of the following conditions70:-
        1. Matching_Photo_Name is null,
        2. Matching_Photo_Name is NOT null, but Photo_Detail is ‘Full’, or
        3. Matching_Photo_Name is NOT null, but Photo_Detail is ‘Medium – Larger’
      • Next, we create the ‘monthly’ jump table that appears before the monthly-segmented elements of the page.
        1. A query is constructed from the Photo_Narratives table, selecting rows for this Timeline (using the Photo_Type from the first column of the main query, which identifies the Timeline). Only level 0 rows are selected. This is so there’s only one entry per month, and to keep the size of the entries down by excluding narratives.
        2. Photo_Type, Photo_Month, Photo_Year and Photo_Narrative are selected, rows being returned in descending date sequence, those without a date appearing at the end as ‘Unknown Date’.
        3. The jump-table has 12 columns71
        4. As many rows as needed are added.
        5. Entries in the table have the Year+Month in the visible text, and the Photo_Narrative as a ‘mouse-over’ pop-up. There’s a hyperlink to the relevant section in the main text.
        6. Entries are padded out with blanks to the end of the row as necessary.
        7. The Photo_Narrative has any bulleting, line feeds and hyperlinks removed for when it appears in the pop-up.
      • The bulk of the page is made up of monthly chunks of photos.

This Note is awaiting further attention72.


In-Page Footnotes:

Footnote 1: Footnote 3: Footnote 13: Footnote 16: Footnote 21: Footnotes 22, 41: Footnotes 23, 24, 45, 47, 50, 62: Footnote 33: Footnote 34: Footnote 38: Footnotes 40, 56: Footnote 42: Footnote 43: Footnote 44: Footnote 46: Footnote 48: Footnote 49: Footnote 51: Footnote 52: Footnote 53: Footnote 54: Footnote 55: Footnotes 57, 59: Footnote 58: Footnote 60: Footnote 61: Footnote 63: Footnote 64: Footnote 65: Footnote 66: Footnote 67: Footnote 68: Footnote 70: Footnote 71:

Previous Version of this Note:

Date Length Title
05/04/2019 10:36:29 5168 Website Generator Documentation - Timelines


Note last updated Reference for this Topic Parent Topic
06/12/2025 01:47:21 1284 (Website Generator Documentation - Timelines (Photos & Blog)) Website Generator Documentation - Control Page

Summary of Notes Referenced by This Note

Awaiting Attention (Documentation) Coxes Farm Coxes Farm (Gardens) Coxes Farm (Pre-Repairs) Coxes Farm (Repairs)
Henry Theo Todman's Blog Website Generator Documentation - Functors Website Generator Documentation - Printable Note Export Website Generator Documentation - Printable Notes
Website Generator Documentation - Reference Functions        

To access information, click on one of the links in the table above.



Summary of Notes Citing This Note

Status: Priority Task List (2025 - November), 2, 3, 4 Status: Summary (2025 - September), 2, 3, 4 Status: Web-Tools (2025 - September), 2, 3 Website - Outstanding Developments (2025 - December), 2, 3, 4, 5, 6, 7, 8 Website Generator Documentation - Backups, 2, 3, 4
Website Generator Documentation - Control Page Website Generator Documentation - Functors, 2, 3, 4      

To access information, click on one of the links in the table above.



Text Colour Conventions

  1. Blue: Text by me; © Theo Todman, 2025



© Theo Todman, June 2007 - Dec 2025.Please address any comments on this page to theo@theotodman.com.File output:
Website Maintenance Dashboard
Return to Top of this PageReturn to Theo Todman's Philosophy PageReturn to Theo Todman's Home Page