Maintenance and validation month!

We spend most of the month reviewing existing functionality like GenDoc, menu, picklists, workflows, and MySQL. Updating deprecated features, making things more stable and fixing edge cases while adding unit tests. We also get the awesome Drag-and-Drop functionality. Even on slow months, this is amazing!

During the database optimization project, we ran into a query that retrieved the fields of a module that could be optimized. This query was executing a subquery to filter duplicated column names which was something that could happen in vtiger CRM. We fixed that years ago, so we decided to add a changeSet to check the database for inconsistencies, in case, anyone still has that error in their database so they can fix it and eliminate the subquery.

Before: Showing rows 0 - 24 (1353 total, 0 in query, Query took 0.0009 seconds.)

After: Showing rows 0 - 24 (1353 total, 0 in query, Query took 0.0004 seconds.)

Half the time!

Note that the new changeSet will do the checks and BLOCK the execution if any error is found. You MUST manually fix the inconsistencies and apply the changeSet again until it passes. In general, the rules of thumb are:

• duplicate fields: delete the field with a higher fieldid because that is what the application query was doing
• different fieldid and custom field name: change the fieldid and recalculate permissions

Evolutivo becomes Drag-and-Drop aware!

• display fields as another field type by using virtual properties in the Denorm widget
• Master Grid
  • check for field dependency if empty
  • insert correct logged-in user
  • load previously created records if the master grid is removed
• Timeline
  • add the map name as a parameter to load the correct view
  • load email form into the Timeline widget
  • read correct data from the API request

We reviewed the code and validated the correct functionality of the extension.

• delete BEFORE reordering else we deleted the wrong entry
• header link
• header scroll missing DIVs
• reordering elements
• copy menu in one query instead of a loop

We also added two CSS variables that permit us to adjust the top-bottom and left-right spacing of the elements. These are useful when you have a lot of elements in the menu and need to make it more compact.

• cbslds-dropdown_item-paddingtb
• cbslds-dropdown_item-paddinglr

We reviewed the code and validated the correct functionality of the extension. We reduced duplicated code and did extensive testing with special character values.

• correct search for existing values to avoid duplicates with different case
• CRUD functionality review and unit tests
• avoid DB error when the picklist name does not exist
• make query secure and reduce query_result
• move ComboUtil function to PicklistUtils and reduce include dependencies
• rename variable and use direct values. function header documentation
• use vtlib picklistValueExists for valueExistsInPicklist
• delete unused code after autoincrement key optimization

• make error messages easier to read and correctly escape HTML
• style Notification Popup by converting href to div for better formatting of large messages
• style Updater to make the installed module message standout
• add getPrimaryKeyOfTable function to database utilities
• add htmlAsteriskSpanWithColor to application utilities
• support user names in copyDashboard.php script
• support all email fields in all modules when opening the email popup selector
• check field metadata table for inconsistencies explained above
• field dependency map to fill in information from the related survey question in the survey path module
• now the Updater mass move fields functionality will add the destination block if it does not exist
• Workflows
  • add CBUUID to the workflow table
  • generate CBUUID and set it
  • set CBUUID when importing new records
  • now that we have the CBUUID we can update the workflow record on import instead of always creating
  • make the workflow tasks more robust. A very important task where we review all the workflow tasks and try to catch edge cases and protect the system from breaking

• EvolutivoFW Standard Formatting
• Documentation: code and wiki.
  • function headers and help messages
  • wiki documentation keeps going
• Refactor and Optimize code
  • SonarQube recommendations, reduce cognitive complexity, change for loops, reduce repeated strings
  • substitute variables for their value, reduce SQL columns, reduce query_result, simplify debug message
  • Optimize field retrieval query
  • add more log messages to the import process
  • eliminate obsolete code in Settings
  • copy menu in one query instead of a loop
  • eliminate unused network call in change password screen
  • optimize code in vtlib_getPicklistValues, UploadDocument and array_inverse (GenDoc) functions
  • eliminate redundant includes in workflow and other parts of the application
  • refactor Emails and Potentials get related record utility functions
• Security fixes and updates
  • update DOMPurify to 3.1.6
  • update ServiceWorker to 7.0.0
  • safely escape Menu URL
• MySQL update
  • MySQL has deprecated the BINARY command which now has to be done with a CAST. We review the whole code to eliminate the hardcoded usage of UTF-8 and convert the BINARY commands to the CAST. This is a complex task that obligates us to review where and why we are using that BINARY so we fix some semantical errors also.
• CI
  • adding the checkout branch feature
  • support playwright testing
• Unit Tests: keeps getting more and more assertions. We increment the code coverage measure in one point!

No changes in the global variables this month.

• Blocks
  • show blocks with no defined session status
  • translate key/header in module for inline related lists
  • use layout block sequence instead of loop sequence
• Calendar
  • avoid double timezone formatting during imports
  • set default color values if none are defined and eliminate a warning
  • set div higher for quick create in popup
• sort mandatory changesets on the initial application load
• Field Dependencies
  • check if the block header element exists before applying actions on it
  • compare with the values of fields instead of types
• recover regression for setting default values when creating from a related list
• Emails
  • strange edge case where the context record is not related to any account/contact
  • we try to send to the email on the record when no other relation exists
• GenDoc
  • add support for DrawFrame and DrawImage directives inside SPAN
  • decode HTML quotes in company fields by reducing query_result
  • force UTF-8 in HTML processing. related to the HTMLENTITES deprecation
  • fix foreach condition with multiple conditions and UTF-8 special characters
  • use cache for tabid and reduce query_result
• Import
  • block the import if the map has structure errors
  • set the correct value on update if the "update empty" property is set
• hide the global discount row correctly in inventory records
• MassEdit better support for file elements
• Web Service: related query count substitution missing space between count and from
• Workflow
  • Launch Workflow task now processes the records if the condition map is not set
  • Update Fields Task: undefined variable $recordid
  • calculate CBUUID for existing workflows on the execution of the CBUUID changeset
  • check for deleted records
  • alert not showing fix when trying to create with no module selected
  • give information in logs when a record is deleted
  • try-catch record retrieval: make workflow more robust
  • unrelate workflow records correctly
  • adding 'is not' operation
• Translations
  • number to string enhancements to support "end cents" in certain languages
  • getJSLang overwrite JSON object only if it does not exist when empty

Another month with a change in the indicators: more code coverage!

July	August

Thanks for reading.