Changes in 1.8.0 Beta1-6.txt

Changes in 1.8.0 Beta 1 - Beta 6  (01 aug 2023 -- 06 sep 2023)
=================================

* Adaptation from Borland Delphi 2006 to to Embarcadero Delphi CE 11.3 (Community Edition 11.3  Alexandria).

  I have abandoned some 3rdParty components/libraries, incorporated new releases of some of them, and patched many of the others.
  In the case of UnRxLib (which I have used to replace RxLibrary), I have made a series of changes to several files, specially
  RxRichEd.pas. Although the changes are marked as such, I include in this commit only the original files, and in the next commit
  I will apply the changes (tested with all the rest of project uploaded in this actual commit).

  Along the way I have identified some errors present in actual version of KNT (and some minor improvements). Some of them have
  been corrected during the process of adaptation to Delphi CE. But in others I have preferred not to correct them at this moment,
  to not mixing them with all the changes necessary to this adaptation (many files, many changes...). I will make specific commits.

  I have decided to drop support for the 'Dart format'. In the case that someone needs to reuse a file in that format, can use
  an older version of KeyNote to convert to normal KeyNote format.

  (More information in doc\README_SourceCode.txt and doc\README_News.txt)

==========
* Find All: Improvements

  - Excerpts from the note of the matches found are displayed, where the searched words are highlighted
 
  - Searches will be much faster
   Now the search is done directly on a string previously retrieved from the rich edit control, instead of continuous call to 
   RichEdit controls. The new search mechanism is much faster.

  - Added new option: "Current node and subtree"

  - The treatment of the options 'All the words' and 'Any of the words' is extended :
   Now, results are shown that include all the words considered, and there may be more than one
   result per node / note.

     - 'All the words': the first word found from the list provided is highlighted, as well
     as those others that appear within the excerpt, to its right. If the last word found
     within the identified match is not shown in the excerpt obtained for the first word
     found, an additional excerpt for the last word is added, where all searched words
     are highlighted.

     - 'Any of the words': Results are created that include, and highlight, all occurrences
     of each of the words in the search term. New results are only added for those words
     that are not visible within the previous excerpt.

  - Global option: Font size in Find All results  (In Advanced | Other tab )

  - Improved the display of node path in search results (when option "Show full node path in search results" is checked)
    I recommend to check the option "Show full node path in search results". The text of the node with the match is now 
    highlighted in the path.
   (Default value for "Show full node path in search results" is now True)

- - - - - 
* Changed: The option "Reverse order" it is used now only with the the option "Show full node path in status bar".
  Node path in search results will always show top to bottom.

* Fixed: The option "Show full node path in search results" was not used. 
  It was always used "Show full node path in status bar" instead. Now both are available and are independent.


==========
* New KNT Links, vinculated to markers, not only to caret position

  When marking a KNT location (Ctr+F6) the app will now insert a small hidden bookmark next to the text
  identified as target. When inserting the link (Shift+F6) the new hyperlink will refer to the created bookmark,
  and will locate it even though all the text could have been relocated (within the same note/node).
  If the marker were eliminated, the caret position would be used, as before.

  This has forced to review multiple aspects of the application, mainly the text search (Find Next and Find All).

  (See a further explanation in the comment before the InsertOrMarkKNTLink method, in kn_LinksMng.pas)
  
 - If we are marking (CTR+F6) the start of a node or note (CaretPos = 0), no hidden marker
   will be created. Link created after will always aim for that position 0.

 - Hint in status bar now shows the number of the new hidden mark created, as a reference.
   It also remember us that we can use Undo to delete a new hidden mark (created by error)

 - Another hint in status bar shows the number of the hidden mark we are pointing it (if any)
  ( Note: hidden mark numbers start by 1 and are specific to each node )
  
  
  * IMPORTANT: a .knt file modified with the last 1.8.0 version, where we used Ctr+F6 and Shift+F6, will
    have little fragments of hidden text besides the text where the marker is located. 
    If the file is opened with an older version:
     - The links will function still ok but using the old behaviour based in caret position.
     - If on the text including this hidden marks is applied Paste font attributes (or the button 'Copy format'),
       or certain edit functions like 'Change Case' or 'White Space' (among others), the hidden characters will
       be made visible.
         For example, if the text "The new Beta is available" has a marker pointing to word "Beta", it could
         change to something like: "The new B1Beta is available". Ascii characters #11 (DC1) y #12 (DC2) 
         will be shown as little squares. (B1: B: Bookmark "1")
     - When searching text that includes hidden marks, the located pattern could be affected.
       For example, if we serch for "new Beta", KNT will not be able to find a coincidence.
     - Also, if text including this hidden marks is copied to other sites inside KNT or outside, the hidden
       marks will be copied also. If the file is then modified with new version, the links that pointed to that
       text could now jump to the new pasted text, if it is upper in the node.
     - In general, that hidden characters could 'emerge' in different situations

  
==========
* Redesigned navigation history mechanism

  - Default navigation is now global and so we can move between the nodes of a note but also between notes
  - Local navigation (traveling only active note's history) is also possible, clicking on the toolbar button with Ctrl.
  - Every jump from an internal keynote link will generate history, not only those that go to another node in the same
    note.
  - Bookmark jumps will also generate history
  - Standard RTF notes (not multi-lvel tree notes) will have its own history navigation
  - Related buttons have been moved to main toolbar.
    Their color and hint indicates if global and/or local history is available in that direction

   (An explanation of the new behaviour is available in the comment before 'NavigateInHistory', in kn_LinksMng.pas)


==========
* New: Added 'KeyNote' as target format in File/Export...

  Now it is easy to create new KeyNote files from current file, with the export mechanism:

  - Including only active note
  - Including only current node and subtree of the active note
  - Including only checked nodes in active note
  - Including selected notes
  - Including all notes (this is equivalent to File/Copy To.., but with the possibility to select the file name)

  In all that cases it is possible to consider only not hidden nodes. For example, if we run 'Find All' with
  'Filter Tree note' option, we can then generate a new .knt with the nodes that include the searched pattern.

  The created .knt file will have the same properties as the original (native, compressed, ..)


==========
* Improved Alarm management window. Revised design and usability


==========
* Added a new property, "Default Zoom": Allows to use a default value other than 100%
  Can be defined as default for the current file or for all the files.

* Changed behaviour of Zoom Text action.
  Usually new zoom value is set by selecting one of the items of the combo. In this case, clicking in one value
  will apply it ALL notes by default. If Ctrl is down while clicking, it will apply only to active note.
  If the new zoom value is typed in the control, pressing Intro will apply it to all notes. Pressing Ctrl+Intro
  will do nothing. To apply a typed zoom value only to the active note, click on the note while pressing Ctrl.

  Double click resets the zoom to <Default Zoom>%, as before. Now it is controlled also by Ctrl key.

  If pressing escape inside zoom combo, active note will gain focus, and no change will be done in any zoom.
  Deleting the combo will be similar to pressing Esc.

  Ctrl will control also the menu options 'Zoom in' and 'Zoom out'.

- - - - - 
* New: View | Alternative Margins

   Allows switching between default (minimum) and defined margins in .ini
  (As suggested in https://github.com/dpradov/keynote-nf/issues/602#issuecomment-1685401368 )
   New options define text margins in pixels:

    MarginAltLeft =80
    MarginAltRight=80

- - - - - 
* New: pressing [Ctrl] + <Hide Tree Panel> will temporarily preserve editor width, reducing the application window width

  When the user hides the Tree Panel pressing Ctrl (usually: Ctr + Shift+F11  or Ctrl + [View | Tree Panel] ) then,
  instead of letting the editor panel grow to the left occuping the width of the tree panel, the application window will
  reduce its width temporarily so that the editor maintain its width and position.
 
  Once shown the Tree panel again, the application window's size and position will be restored.
 
  If, after hiding the tree panel in this way (the window's width is now reduced), the user selects another note, then
  the application will automatically make the tree panel visible and restore the aplication window width, before changing
  to the other note.
 
  Opening onother file or closing the application while the windows's width is reduced, will restore it to it's correct size.
 
  This functionality can be disabled in keynote.ini, with: IgnoreCtrHideTrePanel = 1 

- - - - - 
* Fixed: High DPI displays. The application should scale correctly based on OS settings
  Set 'AppDPIAwarenessMode' in the app manifest as 'GDI Scaling'
  Ref: https://www.softacom.com/blog/windows_application_manifest_in_rad_studio/

- - - - - 
* Improvement in the design of the form Configuration Options
 

==========
* Optimization in file open/save, mainly in native format
   Reduced load and save times, more notably in native format, where it's not needed to
   compress/decompress or encrypt/decrypt the file content.
   
   ** Note: Although reduction in time obtained for compressed format files is much less that for
   native format, I want to recall that it could be more convenient to use that format instead 
   of native format, specially if the file contains images, because of the way images are saved in RTF,
   at least in RichEdit control: images are saved in plain ascii, as hexadecimal values.
   
   I recommend seeing the results of some tests in which I compare the native format with the three options
   of the compressed format: Max, Default, Fastest 
   -> "Comments on KNT file formats.txt"     

- - - - - 
* Fixed possibly difficulty on saving, if file is located in a externally synchronized folder (for example with Dropbox)
    The way backup was realized lead to a little interval where the actual file was moved to the backup folder.
    That little gap could be observed by Dropbox, that detected the movement of the file and asked us if we want to delete the file.
    (more info within the changes: knt_NoteFileMng: NoteFileSave)

  - Minor corrections in the process of saving. Revised some of the messages shown in case of error
  - Revised some hints in several options of the configuration form, related with file save and backup,
    emphasizing the convenience of selecting the option "Backup at regular intervals".


==========
* Fixes and changes in URL actions:
  - Fixed: 'Open in New window' didn't open URL in a new window (worked equal to 'Open')
  - Removed option (in Actions | URL actions ) "Shift key must be held down when URL is clicked"
     (It didn't work. URL Click is not raised when Shift is down)

* Improved click options on hyperlinks
  Now right-click on a hyperlink will always show the prompt window (to edit, modify or open)
  Added a new option to define the action on CTRL + Click

* Fixed: Option "Use system default web browser" was not working 



==========
** Clipboard Capture / Web Clip / Paste external

* Try to offer RTF Format in clipboard whenever possible (from HTML format)
  (as it has been done with 'Paste Special...')

  So far, if we just pasted normal from a web page, without being selected 'Paste external as Plain text'
  we might end up pasting completely plain text, while selecting 'Paste external as Plain text' we
  might paste more or rich RTF (depending on 'Plain text mode'). Confusing.
  This not ocurred when using Clipboard Capture or Web Clip, where conversion from HTML was always needed

  More information in the comment to the procedure PasteBestAvailableFormat, kn_EditorUtils

- - - - - 
* When pasting from HTML (web pages), the current font and size in the active editor will be used as default values,
  instead of the usual Times New Roman 12.

- - - - - 
* When converting from HTML to RTF, on pasting from web pages, last aditional line break will be removed

  In most cases, when pasting from web pages, the RTF obtained after convert from HTML adds at least an extra final line break.
  Now it is removed. This way the result obtained when configuring the divider string (defined in Clipboard Capture or Web Clip),
  is more clear.

- - - - - 
* Fixed: Pasting as plain text could move the caret several characters before its correct position

  In solution to issue #539 (Pasting as plain text moves the caret beyond pasted text when it includes several lines)
  this problem was partially mitigated and caused, on certain occasions, the opposite effect, placing the caret
  ahead of the position that corresponds to it.

  Now I have changed the way the plain text is pasted, and I think it has finally been correctly resolved.

- - - - - 
* Fixed: copying and pasting from certain web pages can produce nothing or truncate some text

  KeyNote will paste from web page as plain text (without any formatting) when detected that conversion from HTML to RTF it is no ok

  It can occur that the conversion to RTF (obtained with TWebBrowser) from the HTML copied to the clipboard from certain pages
  are totally o partially incorrect (even MSWord can't convert correctly that fragments).
  I have detected this problem with the following site, whose html seems to be odd: stackoverflow.com. But it could happen
  with other site/s.

  For example, consider the following URLs and try to copy and paste several paragraphs altogether.
         https://stackoverflow.com/questions/46991729/how-to-load-a-stringstream-with-utf8-characters-to-a-stringlist-in-android
         https://stackoverflow.com/questions/11122197/why-does-memo-lines-use-tstrings-instead-of-tstringlist

  Most of them will produce '<>' or '', others '<<<' or something similar and a few can truncate some text.
  If in this examples we copy (and paste) only fragments of certain paragraphs or just snippets of code, it seems to be ok.


---------------
** Improvements on Clipboard Capture / Web Clip capture

 - Added new token: '%%'

   Added a new token (%%) that delimits a divider string for the second and following consecutive paste
   from the same page (same URL).
   KNT will remember the last URL it was pasted from in the current [Clipboard Capture] session.
 
   This allows, for example, to include the URL and title only before the first fragment we paste from a certain web page. 
   Next (consecutive) fragments of the same URL can include any of the tokens, but normally we will only need a simple 
   delimiter (like '- - - ').
 
   Predefined divider strings have been updated, including this new token and allowing this behaviour.
   It is of course possible to create a custom divider, and you don't have to use the new token in it.

- - - - - 
 - Ensure to include Title for URL  (%S, %s, %U)

   So far, when pasting with Clipboard Capture (and 'Include source URL' selected) or with Web Clip, from a web page,
   only the URL was include, no the title. At the beginning some browsers included that information on the clipboard,
   but currently it does not.
   Now KeyNote NF will try to get the title using WebBrowser (offline, using cache).
   Although it tries to process as little HTML text as possible to get it, the time it takes can depend on the page.
   Normally it wil be very fast, but there are very heavy web pages on witch it can takes a little more time.
   For this reason, the application includes now three tokens to referentiate source URL in the divider string:

     '%S'  insert Source URL (with title)
     '%s'  insert source URL (with title, limited waiting time)
     '%U'  insert source URL (without title)

     With '%S', it will get the title for the URL, without time limit (6 seconds, just for security)
     With '%s', If title is not figured out in less that 2 seconds (normally will be much faster) only URL will be used
     With '%U', KeyNote will only show URL, not title (even though it's already available (*) )

   (*) A small cache will be used for titles, in case you need to paste several times, from the same pages.
    So, even certain delays will only apply to the first capture on a certain URL.

   There is more info on on Clip Web and Clipboard Capture on:
   'Improvements on Clip Web and Clipboard Capture #532' (https://github.com/dpradov/keynote-nf/issues/532)

- - - - - 
 * Title from YouTube page: exception to the normal paste behaviour
  
  From now it seems not possible to obtain the title of a page of YouTube, because the html returned by TWebBrowser
  in this case does'nt contain the final html page.
  So, KNT will not try to guess title if URL begins with https://www.youtube.com
  As a easy shortcut to create a hyperlink that includes the URL and the caption of the video, we can do simply:
  - Select only the caption of the video (just click three times in one word of the video title)
  - Copy Paste (or Web Clip)

  If the text selected includes only 1 line and it hasn't more that 100 characters, then it wil be assumed as the title
  and a normal hyperlink with Youtube URL and that title will be inserted.

  (Note: https://www.smperth.com/resources/youtube/youtube-character-limit/ )

- - - - - 
 * Titles from web pages: convert HTML Ascii codes

  Some titles are available in UTF8, but others express special characters with HTML Ascii codes of the form "&#999;"
  For example, from the page 'https://www.microsoft.com/es-es/windows/windows-11' we can obtain the following title:

   "Descubre Windows 11: la versi&#243;n m&#225;s reciente de Windows | Microsoft"

  Now,it is converted to:
   "Descubre Windows 11: la versión más reciente de Windows | Microsoft"

- - - - - 
 - Added new token: source server/domain (%d)

   If when pasting we are including source, the URL is not visible (*) and the the server/domain is not shown
   in the title (ignoring spaces and case), then the new token '%d' (case sensitive) will show the server in square brackets.
   Some examples:

   'https://martinfowler.com/eaaCatalog/', 'Catalog of Patterns of Enterprise Application Architecture'                   => [martinfowler.com]
   'https://stackoverflow.com/questions/1549145/case-insensitive-pos', 'delphi - case insensitive Pos - Stack Overflow'   => ''
   'https://www.youtube.com/watch?v=r0R6gMw2s44', 'El Círculo de Quintas: Una explicación detallada | Versión 2.0'        => [YouTube]

   (*) URL will be visible, if divider uses %U token or we are pasting in a PlainText note, for example

- - - - - 
 * Revised and improved predefined dividers

- - - - - 
* Adjustment in the option "URLOnly"

  Option only modifiable from .INI, [Clipboard capture]. When enabled (default option) and we copied at most one word,
  that text will be ignored and instead of it, the URL (and title if available) will be shown.

  Improvement in the way it is decided to apply or not this option (depending on text selected)
  Also, URL pasted this way adds line break


---------------
** Clipboard paste / copy / cut

* Fixed bug that could affect the paste action, when determining if normal or plain paste should be applied:
  - When calculating if two strings copied to the clipboard are identical, appying CRC32, the length considered
    were not correct. It should be expressed in bytes, not in characters (different in unicode or WideString)
  - That comparison is considered, for example, to determine if the text is copied from outside KeyNote.
    So, in certain cases the text could be pasted as normal, not plain, even if the option "Paste external
    as plain text" was set.

- - - - - 
* Fixed bug that made that the option "Ignore duplicate clips" were ignored
  Option configurable in:
    Advanced / Clipboard capture
    keynote.ini: [Clipboard capture] TestDupClips

- - - - - 
* Fixed: An exception "Cannot open clipboard: Access Denied" could occasionally be raised
  In that cases the message could be accepted without closing the app, but could be annoying.

- - - - - 
* Fixed: The option "Limited size to", in Advanced/Clipboard ("Maximum length of text to capture, in bytes (when paste as plain text)")
      should only be considered for Clipboard Capture, not for normal plain paste (nor Web clip)


==========
* Several fixes and enhancements related to 'Note properties...'  and 'Default Settings...'

    - Changes in design and functionality in "Default settings.." (F6) and "Note properties.." (F4)
      - Only uses two tabs, "Note setting" and "Tree setting"
      - Allows you to use the current settings of a note tree as the default settings
        for all files or all notes in current file
      - The 'Inherit BG color from active node' property state is shown to clarify the reason for the color being
        displayed for the note (the logic is now the opposite)
      - Some small informative notes are added to clarify issues that can be confusing

    - Modified several captions and hints to avoid possible confusion between color of nodes and color of editor
    - Changed: Moved the option 'Inherit BG color from active node' from 'Tree panel' section to 'Rich Text editor'.

    - Added: Font and BG color can be changed for ALL tree panels at once
      Can be changed from active note (Note | Note properties...) or from default settings (Tools | Default settings...)
      In 'Tree settings' tab: "Apply to ALL tree notes"

    - Fixed: modifying the properties of a note (Note properties..[F4]) could cause font formatting to be lost in certain
      situations (bug existing from version 1.6.5).
      It seems to always happened when changing the properties of a note after the word wrap state on that note had 
      been changed (on the selected node or just by traversing nodes with different WodWrap state).
      Now it is solved and the word wrap option is also included in "Note properties..", not only in "Default settings.."
      Changes in the default font properties of a note should affect only to new nodes, not to existing nodes
      (exception: plain text only notes)

    - Fixed: sometimes the defined default font properties of the note where ignored on new nodes.
      The text written in the new node didn't use the correct font.

    - Fixed: Changes in tree font didn't update correctly
        Font changes in the tree (from 'Note Properties...') were not reflected on it on many occasions,
        precisely those in which we had not modified the background color.
          It was also happening that the font changes, when they were shown, did not give rise, many times,
        to a correct resizing of the height of each item.
        (These problems were of tree refresh. Saving and reopening the file would show up correctly.)

    - Fixed: Sometimes changes from Note Properties (F4) did not modify the status of the file (to modified)
      (eg. changing the editor font size for new nodes), or did not modify the status of the Save toolbar button

* Fixed: Making bold a tree node din't maintain other possible styles of the default TV font


==========
* Fixed: Option "Autoregister file type" now works ok without any special permissions

  Now it is registered for current user, in HKEY_CURRENT_USER. Registered types: .knt, .kne, .knm, .knl
 (HKEY_LOCAL_MACHINE / HKEY_CLASSES_ROOT:
  On the newest Windows versions, you need admin rights to register a file type for all the machine users)

* Fixed: Change in read-only state of a note => save button state
  Setting a note to read-only was marking the file as modified, but was not reflected in the save button


==========
* Fixed: Ensure that all nodes in a RTF tree note are saved in RTF format,
       and all nodes in a plain text only tree note are saved in plain format.

  when dropping a file with plain text into a new node of a RTF Tree, or when dragging and dropping a node with plain content
  (from a tree configured as such) onto a RTF tree, We must ensure that the node's stream is loaded with it's RTF
  conversion, to avoid ending up saving that node's stream content in plain format to the .knt file when saving
  (could be problematic when reading the .knt file).
  Something similar is needed in the opposite case (RTF content into a plain tree), although in this case the only problematic
  case would be to import a plain file dropping it into a new node of a plain tree (more information in the comments to the
  changes).

* Fixed: Text files imported as a new note note could be saved plain, not RTF

  This problem exists in KNT from the beginning, but with the changes included from version 1.8.0
  could be much more problematic, when loading the .knt file

  When importing a file with plain text into a new note, we must ensure that the note's stream is loaded with its RTF
  conversion, to avoid ending up saving that note's stream content in plain format to the .knt file when saving.
  This could occur if we didn't modify the text of that new note before saving the file.


==========
* Modification in RxRichEd to change the way a retry is made when the load of stream detects an error

  It will retry with no TEncoding conversion, setting only SF_TEXT (no SF_UNICODE). The stream will be
  supposed to be in UTF8 format, with BOM. RichEdit will manage it correctly.
  This is how TRichEditStrings.LoadFromStream (in RxRichEd with KeyNote) worked in previous versions.
  Now those symbols incorrrectly converted (with previous versions) wil be ignored, but the rest will be
  shown correctly as UTF8.
  (Resolves issue #610)


==========

* Fixed possible exceptions when no file was loaded

* Changes in log management


==========
* At the moment, the Send Mail functionality will not be included in KNT

  It would be necessary to update it to manage correctly authentication (username and password / smtpAuthXOAuth2 /  smtpAuthOAuthBearer),
  and I think that this is not, probably, a demanded feature of KeyNote.

  Besides, with the update to 'ICS - Internet Component Suite - V8.70', by Francois Piette, it implies
  almost 600kB extra in executable file's size.