-
clean_fetch_results()
: added argumentuseBytes = TRUE
to allgsub()
calls. According to Kurt Hornik, "fetching fails using current versions of R for some contents with non-ASCII characters". -
Added bypass argument
as_is
to methods/functionsget_attachments()
,execute_fetch_attachments()
,fetch_attachments()
; andas_is
functionality toget_attachments()
andexecute_fetch_attachments
as proposed by Pauls Smith to handle non-base64 files.
- changed unit test to Github Actions CI/CD
- removed 'LazyData' line to fix CRAN check issue
- added "\dontrun{}" to
decode_mime_headers()
in order to fix CRAN check issues - fixed bad URLs everywhere (tools.ietf.org/html/... ===> www.rfc-editor.org/rfc/...)
- updated completion msg in all fetch methods to:
"\n::mRpostman: fetch operation is complete.\n"
- typos in
list_attachments()
- typos in
README.Rmd
- updated completion msg in all fetch methods to:
"\n::mRpostman: fetch operation is complete.\n"
- typos in documentation ("Basics" vignette)
- typos in the methods and functions documentation
- updated figures in "Basics" vignette
- incorporates all the changes/updates from GitHub patches
0.9.1-1
to0.9.1-6
.
- modifies the behaviour for assigning names to the local folders. Instead of using the url, now the package uses the "/username/mailbox" as the 'root' directory for saving messages and attachments. For example:
./allanvcq@gmail.com/INBOX/UID1430/
instead of "imap.gmail.com/INBOX/UID1430". This will allow users to better differentiate between local folders, when using different mailboxes in the same server.
- fixed the
pattern8
regex inclean_fetch_results()
.
- changes the default value of the
timeout_ms
parameter to0
, which is the default value in libcurl, meaning that no timeout is specified.
-
added the
clean_msg_text()
function. It will be particularly useful for cleaning and preparing message text retrieved withfetch_body(..., mime_level = 1)
. These texts can be used for sentiment analysis and other text mining tasks; -
added internal functions
decode_mime_text()
anddecode_quoted_printable_text()
. The second is called by the first one; -
renamed the original
decode_quoted_printable
called fromdecode_mime_header()
todecode_quoted_printable_text()
; -
included the
xml2
and thervest
packages as dependencies because of the addition of theclean_msg_text()
.
- added the
mime_level
argument to thefetch_body()
method. Now, the user can choose whether the fetch will retrieve the full body content or an specific one. This is particularly helpful for retrieving clean text parts without inline and regular attachments for instance.
-
adapts to the new return pattern of MS Exchange IMAP servers without losing the compatibility with others;
-
fixed the misbehavior for the attachment fetching of message in message attachments. Now, either
fetch_attachments()
andfetch_text/body()
+get_attachments()
combo can handle this specific case; -
changed the attachment identification to the "name" field instead of the "filename". In very rare cases, the "filename" field identification fails because of encoding errors on the sender's side;
-
incorrect decoding of ordinary words (such as "ResearchGate", for instance) in
decode_mime_header()
.
- The default argument in all
reset_*()
methods now arex
in order to prevent wordy method calls and repetition. The older ones were deprecated in this version.
- added RFC 2047 quoted-printable and base64 MIME header decoder
- added the
decode_mime_header()
function- it is used inside
get_attachments()
andfetch_attachments()
for correctly setting the filenames - it is also exported for the user in order to be used for a header decoding operation after fetching metadata, for example.
- it was necessary to add the {stringi} package as dependency
- it is used inside
-
list_mail_folders()
: fixed the regex related to the hierarchy separator to accept the "|" separator. It was causing a misbehavior in Yandex accounts. -
clean_fetch_results()
(internal): fixed the regex responsible for cleaning the attachment content. It was causing a misbehavior in Yandex accounts. All calls to gsub() in this function now haveignore.case = TRUE
. -
reset_*()
methods: Except byreset_password()
andreset_xoauth2_bearer()
, the other methods were not reflecting the changes in theImapCon$con_params
object, although they were succesfully modifying theImapCon$con_handle
, which was sufficient to work properly. It could cause the user to thing that the changes wouldn't taking any effect.
- The confirmation message in
fetch_attahments()
changed to "\n::mRpostman: the fetch operation is complete.\n"; - Fixed some minor typographic errors in the vignettes and the README file;
- Adjusted the order of messages printed by
rename_folder()
.
- The default argument in all
reset_*()
methods now arex
in order to prevent wordy method calls and repetition. The older ones will be deprecated in version 0.9.1-2; - The
reset_ssl()
method had the name changed toreset_use_ssl()
to better reflect the connection parameter to be reset. - the
metadata
argument infetch_metadata
now isattribute
.
In this version, in order to drastically enhance the package usability, we had to adopt several profound changes with no backward compatibility. We are sorry that these changes will be painful for old users but it will certainly be strongly beneficial even in the short term. The primary update is that mRpostman
now is built on an R6
class and its methods, providing a much more convenient and elegant way of usage. It is structured following an OO framework that works well either with the tidy style using pipes or under the base R approach. The syntax now will be ConnectionObject$method()
, where the ConnectionObject
is created with configure_imap()
or ImapCon$new()
. This completely modifies how the connection handle and other configuration information is passed among the methods/functions in mRpostman
.
This is a summary of the main modifications in the package:
-
The package title has changed to mRpostman: An IMAP client for R
-
All the main functions, except by
list_attachments
and the custom-search helper functions, now are methods of the R6 classImapConf
; -
The way the connection token is passed between the functions has changed. The connection handle is created only inside
configure_imap()
(orImapCon$new()
) and only modified with custom requests inside the methods. As a consequence, the password, username, and other connection parameters are hidden inside the curl handle C pointer, resulting in a more secure token chain. This resulted in changes in every request-functions. They do not useconfig_handle()
anymore, and a call tocurl::set_opt()
is made in every request function so that a custom request is supplied or replaced by a new one in the original handle. -
the argument "by" used in search and fetch functions was replaced by
use_uid
, which is a logical with default value set asFALSE
. This is equivalent to the formerby = MSN
default configuration. -
all functions that returned
invisible(0L)
now returninvisible(TRUE)
list_mailboxes()
->list_mail_folders()
select_mailbox()
->select_folder()
examine_mailbox()
->examine_folder()
custom_search()
->search()
fetch_full_msg()
->fetch_body()
fetch_msg_header()
->fetch_header()
fetch_msg_text()
->fetch_text()
fetch_msg_metadata()
->fetch_metadata()
get_min_id()
->esearch_min_id()
get_max_id()
->esearch_max_id()
count_msg()
->esearch_count()
-
section_or_fields()
-
flag_options()
-
mbox
->name
:-
OBS: type has not changed, still a character string
-
Affected functions:
select_folder()
(former select_mailbox),examine_folder()
(former select_mailbox),rename_folder()
(former rename_mailbox)
-
-
reselect_mbox
->reselect
:-
OBS: type has not changed, still a logical with
TRUE
as default -
Affected functions:
rename_folder()
(former rename_mailbox),copy_msg()
,move_msg()
-
-
by
->use_uid
:-
OBS: type HAS CHANGED from a character string with default value
MSN
to a logical withFALSE
as default. Formerby = "UID"
now is equivalent touse_uid = TRUE
-
Affected functions:
-
all search operations:
search()
(former custom search),search_before()
,search_since()
,search_on()
,search_period()
,search_sent_before()
,search_sent_since()
,search_sent_on()
,search_sent_period()
,search_larger_than()
,search_smaller_than()
,search_older_than()
,search_younger_than()
,search_flag()
,search_string()
; -
all fetch operations:
fetch_body()
(former fetch_full_msg),fetch_header()
(former fetch_msg_text),fetch_text()
(former fetch_msg_text),fetch_metadata()
(former fetch_msg_metadata); -
all complementary operations:
copy_msg()
,move_msg()
,delete_msg()
,esearch_max_id()
(former get_max_id),esearch_min_id()
(former get_min_id),esearch_count()
(former count_msg),add/replace/remove_flags()
-
-
-
flag
->name
:-
OBS: type has not changed, still a character string containing the flag name
-
Affected functions:
search_flag()
,flag()
(custom search helper function)
-
-
string
->expr
:-
OBS: type has not changed, still a character string containing the string or expression to be searched; shifted to the first position of the arguments in the functions/methods.
-
Affected functions:
search_string()
,string()
(custom search helper function)
-
-
section_or_fields
->where
:-
OBS: type has not changed, still a character string containing the name of the message section or the header field in which to execute the search for the informed expression;
-
Affected functions:
search_string()
,string()
(custom search helper function)
-
-
specific_UID
->msg_uid
:-
OBS: type has not changed, still a numeric vector containing message uids
-
Affected functions:
expunge()
-
-
to_mbox
->to_folder
:-
OBS: type has not changed, still a character vector containing the folder name
-
Affected functions:
copy_msg()
,move_msg()
-
-
try_b64decode
->base64_decode
:-
OBS: type has not changed, still a logical with default value
FALSE
-
Affected functions:
fetch_text()
(former fetch_msg_text)
-
-
show_pass
-> removed:-
OBS: This argument is not available anymore.
-
Affected functions:
configure_imap()
-
-
fresh_connect
-> removed:-
OBS: This
curl
argument is not accepted by mRpostman anymore. -
Affected functions:
configure_imap()
-
-
return_imapconf
-> removed:-
OBS: This argument is not available anymore.
-
Affected functions: all search methods.
-
-
in
search_string()
andstring()
:- ``expr
(former string) now is 1st;
where` (former section or fields)` now is 2nd
- ``expr
-
in
esearch_count()
(former count_msgs),esearch_max_id()
(former get_max_id),esearch_min_id()
(former get_min_id):- swapped the order between the arguments
use_uid
(former by) andflag
- swapped the order between the arguments
-
default value of arguments:
reselect_mbox = FALSE
->reselect = TRUE
(former reselect_mbox) inmove_msg()
andcopy_msg()
-
returned object:
-
functions/methods that returned
invisible(0L)
in the previous version, now returninvisible(TRUE)
. Applied to:select_folder()
,create_folder()
,rename_folder()
,get_attachments()
,fetch_attachments()
,expunge()
-
all search functions now return
NA
when there is no match. The previous behavior was to return 0. -
add/replace/remove_flags()
methods now invisibly return the msg_ids in case the user intends to chain any further operation (perhaps expunge) using the pipe.
-
-
default value of arguments:
retries = 2
->retries = 1
in all functions
-
create_folder()
: Create a new mail folder (New IMAP functionality!) -
list_flags()
: List flags in a selected mail folder (New IMAP functionality!) -
fetch_attachments_list()
: Fetch attachments' list without the previously need to fetch a message's text or body -
fetch_attachments()
: Fetch attachments without the previously need to fetch a message's text or body -
reset_*()
: reset one (*) of the original parameter that were informed inconfigure_imap()
.
-
mute
:-
OBS: A logical. If TRUE, mutes the confirmation message when the command is successfully executed. Default is FALSE. In the case of the
fetch_*()
functions, it only has effect whenwrite_to_disk = TRUE
. -
applied to methods/functions:
select_folder()
,create_folder()
,rename_folder()
,fetch_body()
,fetch_header()
,fetch_text()
,fetch_metadata()
,copy_msg()
,move_msg()
,delete_msg()
,expunge()
,add/replace/remove_flags()
,get_attachments()
,fetch_attachments()
-
-
override
:-
OBS: A logical. If TRUE, overrides existent files containing the same name in the local directory. Default is FALSE.
-
applied to methods/functions:
get_attachments()
,fetch_attachments()
.
-
-
xoauth2_bearer
:-
OBS: added the
xoauth2_bearer
parameter for oauth2.0 authentication (libcurl >= 7.70 is required because of bugs in previous versions). In Linux, if you use Ubuntu 20, you should be fine. Versions below this require updating libcurl if the user intends to use oauth2.0 authentication. -
applied to methods/functions:
configure_imap()
-
-
in
examine_folder()
andrename_folder()
:- if a folder was previous selected, the user does not have to provide the folder name
- if the user wants to examine or rename a folder different from the selected folder, he/she has just to inform the folder
name
argument
-
flag
argument in all search functions:- all search methods that have the optional argument
flag
now accept more than one flag as a filtering parameter - this is also applied to the
name
argument insearch_flag
- the
flag
parameter was added to thesearch_string
method
- all search methods that have the optional argument
-
in
get_attachments()
andfetch_attachments()
:override
argument was added and allows to control over the file writing process
-
retry bug fixed: it was causing the loss of the search and fetch
customrequest
when executing a retry + selection operation or when there was a considerable period between two requests given that the second depends on a previous folder selection. When there was a considerable delay between the executions of two commands, the curl handle would establish a new connection to execute the last one, but without the mail folder selection. This was causing an error during the retry or the next IMAP command since the IMAP session would have lost the mail folder selection. This bug was happening mainly when the functions were used under the base R approach- applied to: all request functions/methods, such as search, fetch, mailbox operations (except those that don't need a previously folder selection), and complementary operations
-
fixed bug that was writing metadata .txt files as textUID*.txt
-
Better error handling:
-
An unique internal function called
check_args()
was created to check for the validity of the arguments of all methods and functions, replacing all the cheack_args_* specific functions -
added the
response_error_handling()
function to catch operation/resolving timeout errors and login error as well
-
-
All methods work as wrappers for internal functions with similar names and suffix
*_int
-
Search, Fetch and Complementary functions have a central internal function called
execute_*()
that is responsible for configuring and executing the requests towards the IMAP server -
removed
config_handle()
as the connection token chain has changed -
added
config_conn_handle()
which is called inside theiniliaze()
method of the R6ImapCon
class. -
added
adjust_repeated_filenames()
,serialize_filenames()
andextract_MIME_level_and_filenames()
as helper functions to the new attachments fetch operations -
added
adjust_folder_name()
which is called in almost every request function -
some regex adjustments were made to
fix_search_stripping()
,has_attachment()
, andclear_fetch_results()
-
in
list_server_capabilities()
: changed IMAP command to "CAPABILITY" instead of establishing a new connection -
REGEX structure replacement from
stringr
to base R in:list_server_capabalities()
,examine_folder()
(former examine_mailbox),list_mail_folders()
(former list_mailboxes),select_folder()
(former select_mailbox)
-
fetch_attachments_list()
andfetch_attachments()
are a faster and smart way to respectively list and download messages' attachments. They do not depend on a former fetching step, unlikelist_attachments()
andget_attachments()
. the new methods use BODYSTRUCTURE metadata fetching to identify the attachments, andfetch_attachments()
also issue a FETCH BODY[level.MIME] command to fetch only the parts of the messages that contain the attachments. This prevents unnecessary fetching when users are only interested in attachments. However,get_attachments()
andlist_attachments()
are still available in the package. -
All
fetch_*
methods, andget_attachments()
now use a different path for saving the fetched files. The folder to be created now will have the following structure: imap_server > mail_folder > or . -
Fetched messages that are saved to disk will have different filename structures:
- if
use_uid = FALSE
: body.txt, header.txt, meta.txt, text.txt - if
use_uid
= TRUE: body.txt, header.txt, meta.txt, text.txt
- if
-
Confirmation messages were added to methods:
select_folder()
,create_folder()
,rename_folder()
,copy_msg()
,move_msg()
,delete_msg()
,expunge()
,fetch_*()
,add/replace/remove_flags()
,get_attachments()
,fetch_attachments()
-
A startup message informing about the breaking changes of the version were added by creating the zzz.R file
-
Besides the
mRpostman
Basics vignette, two more were added: Migrating old code to the new mRpostman's syntax, and IMAP OAuth2.0 authentication in mRpostman
-
get_attachments()
function:- added support to inline attachments: argument "content_disposition"
- changed REGEX for retrieving text and filenames
-
check_args_get_atatchments()
helper function:- added support to inline attachments: argument
"content_disposition"
- added support to inline attachments: argument
-
list_attachments()
function:- added support to inline attachments: added a column
"content_disposition"
to output - changed REGEX for retrieving text and filenames
- added support to inline attachments: added a column
-
check_args_list_attachments()
function:- added helper function
check_args_list_attachments()
- added helper function
-
has_attachment()
helper function:- added support to inline attachments
-
loop_fetch_msg_*()
functions group:- changed error message
-
DESCRIPTION FILE:
- changed title to IMAP Toolkit
- changed Description field
-
README.md:
- item 6) Attachments: added mention to inline attachments
- changed description
- added Outlook - Office 365 configuration example
- changed all examples from UC Riverside to Kansas State University (sorry UCR!)
-
Basics Vigntte
- changed description
- added Outlook - Office 365 configuration example
- changed all examples from UC Riverside to Kansas State University (sorry UCR!)
-
get_attachments()
:- added new note to explain Content-Disposition types
- added reference to RFC2183
-
check_args_get_attachments()
:- added new argument
"content_disposition"
- added new argument
- removed garbage script
environments_tests.R
-
get_attachments()
function:- added
get_attachments()
- added helper functions:
has_attachment()
,check_args_get_attachments()
- added
-
list_attachments()
function:- added
list_attachments()
- added
-
loop_fetch_msg_*()
functions:- error msg: fixed from
examineMailbox()
toexamine_mailbox()
- fixed "forbiden[_]char[s]" in
loop_fetch_full_msg()
- changed forbiden_chars to
[\\/:*?\"<>|]
, considering WIN-*NIX-OSX cases
- error msg: fixed from
-
fetch_full_msg():
- added warning message related to
get_attachments()
whenkeep_in_mem = TRUE
fetch_*()
functions:
- added
attr(msg_list, which = 'mbox') = new_imapconf$mbox
to all -- to be used byget_attachments()
list_server_capabilities()
function:
- converted regex match/extraction from stringr to base R
- examples:
- fixed typo - from "configureIMAP" to "configure_imap" in the following functions examples:
list_mailboxes()
,list_server_capabilities()
- changed all imaps://imap.gmail.com server examples to a generic one: imaps://your.imap.server.com
-
DESCRIPTION FILE:
- changed title to IMAP Toolkit
- changed Description field
-
README.md:
- changed dev installation example from
devtools
toremotes
because the former depends oncurl
- fixed typo - from "configureIMAP" to "configure_imap" in README.md
- fixed typo - two AOL examples instead of one AOL and one Yahoo configure_imap example
- added Yandex
configure_imap()
example
- changed dev installation example from
-
basics.Rmd vignette:
- fixed typo - from "configureIMAP" to "configure_imap" in README.md
- fixed typo - two AOL examples instead of one AOL and one Yahoo configure_imap example
- added Yandex
configure_imap()
example - removed figures (already appear on README.md)
-
count_msgs()
:- changed title
-
loop_fetch_msg_XXXX()
functions for fetching msgs:- added
select_mailbox(imapconf = new_imapconf, mbox = new_imapconf$mbox)
inside the tryCatch in the while loop -- it prevents errors especially when messages have larger attachments taking too much time to fetch. In some IMAP servers (such as Yandex) it may lose the mbox selection. Other alternatives: set a largertimeout_ms
inconfigure_imap()
.
- added
-
list_mailboxes()
:- REGEX for cleaning mbox names from
list_mailboxes()
was modified to deal with the return of Yandex IMAP server: from.*\" \"*(.*?)\\"\r\n' to '.*\" \"*(.*?)[(\\"\r\n)|(\r\n\\*)]
- REGEX for cleaning mbox names from
-
function
configure_imap()
:@param retries description
typo: from "retires" to "retries"
-
internal function
conifg_handle()
:- removed
@family config
, so it does not appear as "see also" inconfig_IMAP()
- removed
-
function
flag_options()
:- updated
@note
to be more clear
- updated
-
all internal functions from
@family check args search
:@param esearch
typo: fromcode{1,2,3,4,5}
to\code{1,2,3,4,5}
-
pkg logo:
- improved logo with background transparency - better looking favicons
-
README.md:
- changed section "First Things First" to "Allowing Less Secure Apps Access"
- changed function name patterns to those specified in the tidyverse style guide
- changed return from mailboxes operation functions and some miscellanea functions:
- now
select_mailbox()
,rename_mailbox()
,copy_msg()
,move_msg()
,delete_msg()
,expunge()
,add/remove/replace_flags()
outputs are invisible and only returnimapconf
or a list (imapconf+msg_ids).
- now
- changed package logo