Rob Kraft's Software Development Blog

Software Development Insights

Archive for the ‘Code Design’ Category

An Example of Upgrading Code From Average to Excellent

Posted by robkraft on June 13, 2021

Average programmers get the job done.  Excellent programmers get the job done too, but the code of excellent programmers lasts longer and is easier to change to meet future requirements.  Below is an example of upgrading average code to excellent code.

Our team decided an application could benefit from caching some data in our .Net application.  We selected a caching framework with a good reputation and implemented it in one of our business objects.  Here is the code:

private static IAppCache _cache = new CachingService();
public static IList<Zone> AllZones
{
    get
    {
        return _cache .GetOrAdd("zones", () => ZoneManager.LoadAll(), new
            DateTimeOffset(DateTime.Now.AddHours(1)));
    }
}

The code worked and we noticed a performance improvement in our Zones class.  The code was easy to implement.  We decided we would implement something similar in several other classes.  However, the above code is just “average”.  What I mean by that is while “it works”, it could be made better.  Allow me to talk through many of the considerations carefully.

  1. If the code solved a problem and we desperately needed to get it into production right away to keep from losing thousands of dollars per hour, then we should probably deploy it right away.  The business need is more important than the developer’s preference for robust code.
  2. If this was the only place in the application that would need this type of caching logic and it is working well, then the code is probably “good enough” and needs no further consideration or enhancement.
    1. In fact, if this code works as is, and would never need to be modified again, spending additional time to enhance the code (from good to excellent) would actually be a failure.  It would be a waste of time, assuming that the developer(s) involved had alternative activities they could engage in that would add more value to the software or the developers skills than refactoring the above code.
    2. If the above code exists specifically for a short-term project, and it will be deleted and no longer used within a week, and refactoring would not improve performance, but would just make the code easier to maintain in the future, but the code really has no future; then refactoring the code is pretty much a waste of time.

In our application, the above caching code was the start of a pattern for caching we wanted to implement in other business objects.  Whenever you are starting some code that will be the pattern by many developers across the code base, you probably want to think about improving the code in the following ways:

  1. Make the code as easy to implement in each place as possible.  That means:
    1. minimizing the references you need to add for each implementation;
    2. minimizing the properties and methods and supporting features you have add each time you use it;
    3. minimizing the code changes you need to make as you copy/paste it from one place to another.
  2. You want developers to fall into the Pit of Success.  This means you want to make it easy for everyone using the pattern to get it right.  The fewer changes they have to make, and the more obvious those changes are, the more likely the other developers will successfully implement similar code in other places.

We were aware that the framework we used for caching could change.  We started with a framework called LazyCache that uses the IAppCache interface and CachingService, which you can see in the code above.  But what would happen if we decided we needed a different caching framework in the future?  We might have to go back to each business object and change the interface we used, and probably replace CachingService.  Can we refactor this code to make it unlikely the code in the business object needs to change if we decide to use a different caching technology such as Redis?  I believe the answer is ‘yes’.  Our goal then becomes the following:

  1. Move all the code specific to our caching technology (LazyCache in this story) into a separate class, so that our business objects are totally unaware of the technology used for caching.
  2. Pass all the values and method needed for the caching feature from the business object to the new caching class we create.
  3. Minimize the things we need to pass to the caching framework to make the code easy to implement in each business object.
  4. Write the caching class in a way so that it also does not need to know much about the business objects that it is caching.  Make sure that it does not need to reference those business objects.
  5. Write code so that the caching class does not need to be changed when it gets used by additional business objects.  It can become a “black box” to future developers.

The revised code in the business object looks like the following:

public static IList<Zone> GeoZonesFromCache
{
    get
    {
        return CacheManager.GetOrAdd(CacheManager.ZONES, () => ZoneManager.LoadAll());
    }
}

Notice that it no longer has any reference to the caching service.  This should allow us to change the way the data is cached from using LazyCache to some other Caching framework including caching services like Redis Cache without needing to modify each business object.  We pass the minimum information which includes the name of the cache (which is a ReadOnly string named CacheManager.ZONES), and also the function to run (ZoneManager.LoadAll) if the cache is not already populated.

We did have to write more code in our CacheManager.  Here is the code:

using LazyCache;

namespace MyApp
{
    public static class CacheManager
    {
        private static DateTimeOffset GetCacheDuration(string cacheType)
        {
            return new DateTimeOffset(DateTime.Now.AddHours(1)); 
        }

        private static IAppCache _cache = new CachingService(); 

        public static IList<T> GetOrAdd<T>(string cacheType, Func<IList<T>> itemFactory)
	{
	    return _cache.GetOrAdd(cacheType, itemFactory, GetCacheDuration(cacheType)) as IList<T>;
	}

        public static readonly string ZONES = "zn";
    }
}

In the code above I include a using statements, for LazyCache.  This is the only class in the entire application that references LazyCache.  If we decide we want to replace LazyCache with Redis cache or some other cache then this one class should be the only place in our app where we need to make a change.

Is the code that we wrote perfect?  The answer to that is definitely ‘Maybe’.  We can’t know if existing code is perfect until time has passed and we determine if it met our needs.  I believe that the code has to meet these criteria to be considered perfect:

  1. If the code never needs to be changed, then it was perfect,
  2. If the code does need to be changed, but it is easy for the developer to change in order to adapt to a requirement change, then it could still be considered perfect when originally written, especially if the changes are isolated to the CacheManager class and don’t need to be made in each business object that uses it.
  3. If changes are needed in each business object, but the changes are easy to implement and were not foreseen when the code was first written, then you could still consider the original code to be perfect.  For example, perhaps some business objects need to pass the cache duration into the CacheManager service.  Assuming that code is easy to implement, and it was not originally expected and coded for, then the original code could still be considered perfect. 

Some of you may identify that improvements to the CacheManager are still possible, and that is certainly true.  One improvement would be a change to make it easier to write unit tests for the business objects using CacheManager.  The code I have above is hard-coded to use the “CachingService”.  It could be helpful if the “CachingService” could be mocked away in unit tests, especially if you replace the CachingService with Redis Cache.  Fortunately, given that the caching code is contained within a single class, it would be fairly simple to change the CacheManager to use an IOC framework.  I won’t delve into that here other than to point out that part of writing excellent code might include the ability to unit test that code you have written.  I will also point out that the refactored code (from “Average” to “Excellent”) makes the writing of unit tests to test the CacheManager itself easier.

You may also notice that the code contains no error handling.  I did not include error handling because it would add clutter that is irrelevant to the topic of this article, and also because some programmers may prefer errors to bubble up the call stack to be caught elsewhere in the application.

To recap this article, if you want to go from being an average developer to an excellent developer in a scenario like this you should do the following:

  • Before writing code, first identify if there is already a solution to the problem in your code base and determine if you can reuse the same solution.
  • When you are writing code, ask yourself if part of the code could be re-used in other places in the application, or perhaps in the future.
  • Take some extra time to write the code in a way to make the pattern easy to implement correctly in every place it will be implemented (or at least in many of the places).
  • Write the code in a way to minimize the need for modifications to each business object if changes to the service being implemented are needed.
  • Make the places where the pattern is implemented unaware of the details of how it is implemented.  In other words, the business objects have no idea what caching technology is used and won’t need to be altered if that caching technology is changed.
  • Make the pattern unaware of the specifics of the objects that are using it.  In other words, the CacheManager does not need to be able to reference and understand the business objects that are calling it.  There is no dependency there.
  • Don’t pass hard-coded strings.  Use an Enum or string constants.  This allows you to change the string value in a single place if you ever need to do so, and, more importantly, insures there are no string typos made.  In the class above, this was done for the CacheType (CacheManager.Zone).  We could have passed a literal string “zn” into the CacheManager, but we anticipate adding logic, perhaps a switch statement based on the cacheType to obtain the duration for the cache, which means we would have a second place (the switch statement) also with a hardcoded string of “zn”.  By using a readonly variable we eliminate the risk of typos for that string.

This is not an article about caching.  This is an article about taking code that is average and making it better, using the implementation of some caching logic as an example.  Some of you may notice that our change to make the code better is also an example of “Separation of Concerns”.  This is true, but the article is not about “Separation of Concerns”, it is simply about writing  better code.  Finally, I will point out that this example is also an example of the “Façade” pattern, but again, this is not an article about the “Façade” pattern, just an article about better code.

I hope this article helps some average developers along their journey to becoming excellent developers.

Posted in Code Design, CodeProject, Coding | Leave a Comment »

Programmer’s Parable – Gathering Requirements Up Front

Posted by robkraft on June 12, 2020

 

conversation-1262311_640The business owner asked the software developers to build a new web application for a single client.  She said that the web app needed to be really fast.  She told the developers this could be the first version of a new product that the company would build upon for years to come.

The developers were excited to work on something new.  They wanted to put their Agile skills to use and see if they could deliver the product within the expected two months while hitting their quality and performance goals.  They wondered what the product owner had in mind for the future beyond the first two months, but she was not readily available so they figured they would follow the Agile practice of focusing on one thing and getting it completely done before thinking about future features.

The team built a great site on time and within budget.  They cached as much data as possible to make it really fast.  The business owner was pleased.  The customers was pleased.  The developers were pleased because they had made everyone happy.  But the cloud IT manager was not pleased.  He noticed the price charged to the client barely covered the hosting cost of running the application in the cloud.

dancing-156041_640

At the party to celebrate the success of the web site, the cloud IT manager asked the business owner if she had a plan to reduce the hosting costs so that the return on investment (ROI) could be earned more quickly.  She laughed and said, “Of course I have a plan to improve ROI, but it is not by reducing hosting costs.  Now that we have one client on board, I’m going to have the team add dozens more clients to the same web site.  The incremental cost of adding each new client should be trivial, just like it is with our other products.  Once we have more clients, the hosting costs will seem trivial!”  She walked away to talk to someone else without recognizing the doomed expression upon the faces of the developers that overheard the conversation.

The Scrum Master saw the developers looking sad.  He asked them why they looked sad.  A developer replied, “She said she wants us to add more clients to the application.”, to which the Scrum Master replied, “Yes, I know.  It is going to be the top priority in our queue for the next iteration.  But why does that make you sad?”  The developer took a deep breath and said, “Because the only reason our web app is so fast is because we cache so much data on the server to avoid database calls.  The data cached is unique for the client.  We didn’t know that we would be expected to support multiple clients on the same web app.  We are going to have to redesign the entire app to support this.  Plus we don’t know if we can make it as fast as it is now.  If we had planned for the next release when we were developing the first version, we would have been able to avoid the complete rewrite that is now needed.”

checklist-150938_640

Analysis

Is it Agile if you engage in Big Design Up-Front (BDUF)?  The answer to that question is to point out that this is the wrong question.  In software, the question should never be, “Is it the Agile way?”.  The question should be, “What it is the best way?”.  Also, in this parable, the problem is lack of future requirements, not lack of future design.

In hindsight, knowing there would be future requirements, the team should have gathered some of those requirements up front.  Gathering requirements and creating designs for future releases is not “anti-Agile” and does not violate any rules of Agile development.  Getting requirements and thinking about design early is good.  You should only question “the amount of time” you spend gathering requirements and creating designs for future releases; not whether or not you do so at all.  If the time you spend on requirements and design for a future release moves beyond hours, beyond days, and into weeks, then you should become concerned that the time you are spending will become waste.  When that time is measured in a few hours, it is probably well worth it.

https://en.wikipedia.org/wiki/Big_Design_Up_Front

Posted in Code Design, Coding, Process, Project Management | Leave a Comment »

Programmer’s Parable – The Last Minute Decision

Posted by robkraft on June 7, 2020

accountant-2252316__480

The software developers looked forward to making the desktop software available for download tomorrow.  They had worked hard to create a high quality, robust program to support customer needs for many years to come.  The business owner stopped by and asked if they had included the discount on purchases for his brother’s company into the software.  The developers looked surprised.  They did not know about this requirement.  The team lead admitted, “I’m sorry, I forgot about it.  You mentioned it to me when we had lunch a few months ago but I did not write it down.”  The owner, said, “My brother will be one of our first customers, can you add this feature today before we release?”  One team member, a mid-level software engineer, spoke up, “Yeah, I think we can make a few simple changes to show a discount in the desktop user interface (UI) only for your brother’s specific account number and it should work fine.”  The response seemed to please the owner, but the software architect replied to the mid-level engineer, “I’m not so sure.  That may create new problems.”  He continued, addressing the product owner, “We will discuss it and get back with you.”

So, the developers held a meeting to discuss the problem.  They estimated they could modify the desktop program in a few hours as the mid-level engineer suggested to solve the problem in the short term, but what if additional customers needed a discount?  A programmer would need to make another code change and they would need to publish a new version as a patch.  And how would that logic get included into the mobile app they intend to release later that year?  They would have to code a similar workaround there.  They would have to maintain the quick fix code in several places.  Putting a quick fix into the UI introduces risks that the logic won’t be applied elsewhere when needed; it would also be harder to automate unit tests for the new logic; and it would require programmer intervention for future similar changes.  Another option, and the one the architect preferred, was to add a column to a table to indicate the discount, change the stored procedures and business objects to handle the discount; and also create a new user interfaces to allow the product owners to configure discounts without programmer intervention.

freelance-1989333__480

The developers made quick estimates and presented these three options to the product owners to make a decision:

Option 1: A quick fix.  They estimated a few hours to complete coding and testing the work.  The benefit would be releasing on time.  The drawback would be a design that contained risks that this business rule is not maintained across the entire code base and that additional patches to the code would be needed if the hard-coded discount changed or needed to be applied to other customers.  Another drawback is that the software architect did not favor this approach and there were already concerns that he was getting frustrated with the company and might choose to quit.  The mid-level engineer however favored this approach because it would work, and it would get the product to the customers faster so they could be happier faster.

Option 2: Delay the release for two weeks to allow the team to improve the design to support discounts that could be configured by business users without future changes to the code.  The benefit would be a design that is more adaptable to changes in product discounts in the future and that the product discount logic could be shared with the mobile app.  The drawback is that they would have to tell the customers the product release is delayed for a few weeks.

Option 3: Perform the quick fix, but immediately start working on the improved design to allow users to configure the discounts.  The benefit would include releasing the product on time, and the architect could get the good product design he was looking for.  However, the drawback to this approach is that it would take four weeks to roll out the product fix with the new design because extra work would be needed to version the API due to the changes required to it and also to migrate the database changes from one version to the next as well as to perform all the upgrade and migration testing.

Which option would you chose?

There is no correct answer, but one choice may be better than another depending on the situation.

If the product is a Christmas promotion application for the year 2020, then the quick fix of option 1 is likely best assuming the program will only be used for a few months before it is discarded.

software-4429957__480

If the product will help its users start making or saving tens of thousands of dollars every day they use it, then you probably want to get it to them as quickly as possible, thus the quick fix of option 1 or option 3 is likely the best option so the gains can be realized even though a higher cost is placed development.

If the development team has other valuable applications to develop, then perhaps you should choose option 1 instead of option 3 because the return on investment (ROI) of another application is greater than the cost and eliminating the risks than come with option 1.  Perhaps the business has another product that will earn a $100,000 per day once deployed.  Should the developers spend a few weeks or a month making the first product better when it only brings in $5,000 per day instead of starting work on the second product that will earn $100,000 per day?  If the profit earned daily is high, then you should probably lean toward option 1.  If the profit earned daily is low, then you should probably lean toward option 2, assuming the application will be around for years to come.  If developer costs are low; particularly if your developers have nothing else they could be doing with their time, then option 3 is probably a good choice.

board-3700116__480

If developer morale is important and if the decision seems to have a big impact on developer morale, then perhaps you should accept the approach they recommend in order to boost their morale and increase their sense of ownership in the product.

We, and here I am speaking as a developer myself, often desire to fully complete a product or feature we were working on.  There is satisfaction and value in doing so.  But sometimes maybe it is best to leave some code and applications partly sub-optimal because there are things we could be doing with our time that are more valuable to the company.

In software development we often have many factors to consider in making product decisions.  Rarely are the costs and benefits clearly measurable.  Even time spent in analysis causes a delay in execution and therefore contributes to waste and increased costs, so it is sometimes counter-productive to spend a lot of time in analysis.

 

 

Posted in Code Design, Coding, Process, Project Management | Leave a Comment »

C# .Net SQL Injection Detection – Especially for Legacy .Net Code

Posted by robkraft on March 4, 2020

When you have an existing .Net code base full of SQL statements, and you want to reduce the chance that there are SQL injection risks in the code, you may decide to perform a review of every SQL statement in order to confirm that they are all coded correctly; or you may hire another company to do this for you. But one problem with this approach is that the code is only “SQL Injection Free” from the moment the review is completed until people start modifying and adding to the code again.

What you should strive for is a way to make sure every past and future SQL statement gets tested for SQL Injection risk before it runs.  That is what this sample code provides you.  If you follow the patterns described here, I believe you can significantly reduce the risk that your code has bugs leading to SQL Injection and it will stay that way going forward.

Using the Decorator Pattern to Provide a Place To Add SQL Injection Detection

The primary technique I recommend in this article for adding SQL Injection detection into your application is to stop using the .ExecuteReader and .ExecuteNonQuery methods.  Instead, use the Decorator pattern to create your own method to be called in place of those two, and that method will include code to do some SQL Injection detection.

Replace:

SqlDataReader reader = command.ExecuteReader();

With 

SqlDataReader reader = command.ExecuteSafeReader(); //provided in sample code

The sample code provided behaves like the Proxy pattern in that it will make the actual call to the database after finding no SQL Injection risk.  The benefit of this approach is that you can then regularly scan your entire code base for the use of .ExecuteReader and .ExecuteNonQuery knowing that there should be no cases of those methods, other than the exception cases you expect.  Thus you can be sure that the majority of your code is running through your SQL Injection detector.

Another benefit of using the Decorator pattern to implement SQL Injection Detection is that you can also easily add other features such as:

  • Logging every SQL that is executed
  • Logging and blocking every SQL that is a SQL Injection risk
  • Altering every SQL on the fly.  One scenario where this could be helpful is that if you renamed a table in the database but had a lot of SQL that needed to change.  You could possibly add a find/replace to every SQL on the fly to change the table name, allowing you more time to find and correct all stored SQL fragments with the old table name.
	public static SqlDataReader ExecuteSafeReader(this SqlCommand sqlcommand)
	{
		if (!sqlcommand.CommandType.Equals(CommandType.StoredProcedure))
		{
			var sql = sqlcommand.CommandText;
			//Options: You could Add logging of the SQL here to track every query ran
			//Options: You could edit SQL - for example if you had renamed a table in the database
			if (!ValidateSQL(sql, SelectRegex))
				return null;
		}

		return sqlcommand.ExecuteReader();
	}

The SQL Injection Detection Code

Warning!  This does not detect all forms of SQL Injection, but it will detect most of them.  Here is what causes the class to throw an exception:

  • Finding a single apostrophe (single quote) that does not have a matching single apostrophe (single quote)
  • Finding double quotes that do not have a matching double quote.  This is only needed if the SQL Server has SET QUOTED_IDENTIFIER OFF.  However, you may also want to use this if your database is MySQL or some other DBMS.
  • Finding a comment within the SQL
  • Finding an ASCII value great than 127
  • Finding a semicolon
  • After extracting the strings and comments, finding any of a specific configurable list of keywords in a SELECT statement such as DELETE, SYSOBJECTS, TRUNCATE, DROP, XP_CMDSHELL

The code is written to be easy to change if you don’t want to enforce any of the rules above, or if you need to add similar rules because you have a special scenario or a DBMS besides SQL Server.

The code uses the regex [^\u0000-\u007F] to reject the SQL if it contains any non-ASCII characters.  This works for the applications I have written, but may need alteration for non American English language support.

The code also uses regexes to check SQL statements for undesirable keywords.  One regex is for SELECT statements and therefore blocks them if they contain INSERT, UPDATE, or DELETE.  Other keywords that may indicate a SQL Injection attempt are also rejected and that list includes waitfor, xp_cmdshell, and information_schema.  Note that I also include UNION in the list; so if you use the UNION keyword you will need to remove that from the list.  UNION is frequently used by hackers attempting to perform SQL Injection.

private static void LoadFromConfig()
{

	_asciiPattern = "[^\u0000-\u007F]";
	_selectpattern = @"\b(union|information_schema|insert|update|delete|truncate|drop|reconfigure|sysobjects|waitfor|xp_cmdshell)\b|(;)";
	_modifypattern = @"\b(union|information_schema|truncate|drop|reconfigure|sysobjects|waitfor|xp_cmdshell)\b|(;)";
	_rejectIfCommentFound = true;
	_commentTagSets = new string[2, 2] { { "--", "" }, { "/*", "*/" } };
}

SQL Server supports two techniques to comment out SQL code in a SQL Statement, two dashes, and enclosing the comment in /* */.  Since it is unlikely that developers write SQL to include comments, my default choice is to reject any SQL containing those values.

Exactly How Is The SQL Injection Detected?

There are basically three steps in the SQL Injection detection process.

First, the code checks for any ASCII values above 127 and rejects the SQL if one is found.

Second, the code removes all the code withing strings and comments.  So an SQL that starts out looking like this:

select * from table where x = ‘ss”d’ and r = ‘asdf’ /* test */ DROP TABLE NAME1 order by 5

becomes this:

select * from table where x = and r = t DROP TABLE NAME1 order by 5

Third, the code looks for keywords, like “DROP” and “XP_CMDSHELL”, in the revised SQL that are on the naughty list.  If any of those keywords are found, the SQL is rejected.

Formatting Methods included in the SQLExtensions Class

The SQLExtensions class provides additional methods to help your coders reduce the risk of SQL Injection.  These methods help coders format variables in SQL when doing so with a parameter is not an option.  The most useful of these methods is FormatStringForSQL and it could be used as shown here to enclose a string in SQL quotes as well as replace any single quotes contained within the value with two single quotes.


string sql = "select * from customers where firstname like " + nameValue.FormatStringForSQL();

Another advantage of using a method like this is that it makes it easy for you to change how you handle the formatting of strings everywhere within your code if you discover that you need to make a change.  For example, perhaps you decide to move your application from SQL Server to MySQL and therefore that you also need to replace double quotes in addition to single quotes.  You could make the change within this method instead of reviewing your entire code base to make the change one by one for each SQL.

Custom .Net Exception Class

I also provided a custom Exception primarily to show how easy it is to implement custom exceptions and because I think it is useful for this extension class.  This provides you more flexibility for handling exceptions.  You can catch and handle the exceptions raised specifically due to SQL Injection risk different than exceptions thrown by the underlying ADO.NET code returned from the database.


[Serializable]
public class SQLFormattingException : Exception
{
	public SQLFormattingException() {}

	public SQLFormattingException(string message): base(message) {}
}

The Rules For Detecting SQL Injection are Configurable

I made enabling/disabling configuration of the SQL Injection detections easy to change so that you could import those rules at runtime if desired so that different applications could have different rules.  Perhaps one of your applications needs to allow semicolons in SQL but the others don’t.  It is a good practice to implement the most stringent rules you can everywhere you can.  Don’t implement weak SQL Injection detection rules everywhere because a single place in your code needs weaker rules.  The rules are “Lazy Loaded” when needed, then cached, to support the ability to change them while an application is running by calling the InvalidateCache method provided.

Below is an example of one of the rules.  You can configure your code to reject the SQL if it contains SQL Server comments.


#region RejectComments Flag
private static bool? _rejectIfCommentFound = null;
public static bool RejectIfCommentFound
{
	get
	{
		if (_rejectIfCommentFound == null)
		{
			LoadFromConfig();
		}
		return (bool)_rejectIfCommentFound;
	}
}
#endregion

Steps To Implement and Use This Code

I suggest you take the following steps to implement this class:

  1. Get the SQLExtensions.cs class file into a project in your code base. You will also need the CustomExceptions.cs class file.  The program.cs just contains a sample usage and there is also a UnitTest1.cs class.
  2. Comment out all the lines in ReallyValidateSQL except for the “return true”
  3. Do a find and replace across your entire code base to replace ExecuteReader with ExecuteSafeReader
  4. Compile and test.  Your app should still work exactly the same at this point.
  5. Review the Customizable Validation Properties and decided which ones you want to implement, then uncomment the lines you commented out in ReallyValidateSQL
  6. Decide if you need to and want to replace dynamically constructed SQL in your application with any of the four FormatSQL… extension methods provided.
  7. Provide me feedback

MIT FREE TO USE LICENSE

This code has an MIT license which means you can use this code in commercial products for free!

A link to the source code example is here: https://github.com/RobKraft/SQLInjectionDetection

Posted in Code Design, CodeProject, Coding, Security | 2 Comments »

Use A Google Sheet To Send Reminder Emails To Your Team For Free

Posted by robkraft on May 26, 2019

A lot of small teams could use reminder emails when it is time for a team member to perform a task, but there are not a lot of products where you can easily set up reminder emails for team members for free.

But you can do it easily with a Google Sheet.

Building on the work of others I created this little script you can copy/paste from https://github.com/RobKraft/GoogleSheetBasedEmailReminders

Open the Script Editor from the Tools menu of your Google Sheet and paste this script in.  The code is simple and documented if you desire to change it.

Then set up 4 columns in your google sheet.  Make row one headers for the 4 columns:

  • Column A: Email Address – this is a single email address or comma separated list of email addresses to send to
  • Column B: Reminder Begin Date – this is the date at which the reminder will start going out daily Column
  • C: Subject – This is the subject of the email
  • Column D: Email Body – This is the body of the email. Also the code adds some extra stuff to the body of the email.

You also need to create a trigger in your google sheet.

To do this, select the Edit menu from the script menu and select Current Project Triggers. You may need to give your project a name and save it at this point. Add a trigger. At the time of this writing in May 2019, you would need to set these values for your trigger:

  • “Choose which function to run” – probably sendEmails
  • “Choose which deployment to run” – probably Head
  • “Select event source” – Time-driven
  • “Select type of time based trigger” – Day Timer – for once per day
  • “Select Time of Day” – During what time frame do you want the trigger to run. (GMT Time)

That is it – save that trigger and it is all yours.  Set up an email to yourself to test it all.  All the emails will be sent from your own @gmail.com account.

Just for fun, I include the script code here that is also in the repo:


function sendEmails() {
  //Set up some variables
  var startRow = 2; // First row of data to process
  var numRows = 100; // Number of rows to process
  var currentDate = new Date();
  var currentYear = currentDate.getFullYear();
  var currentMonth = currentDate.getMonth() + 1;
  var currentDay = currentDate.getDate();
  var emailSubjectPrefix = 'Reminder: ';
  var urlToGoogleSheet = 'https://docs.google.com/spreadsheets/????edit#gid=0';

  var sheet = SpreadsheetApp.getActiveSheet();
  // Fetch the range of cells A2:D100
  var dataRange = sheet.getRange(startRow, 1, numRows, 4);
  // Fetch values for each row in the Range.
  var data = dataRange.getValues();
  for (i in data) {
    var row = data[i]; //Get the whole row
    var emailAddress = row[0]; // First column of row
    if (emailAddress != "") //If there is an email address, do something
    {
      var eventDate = new Date(row[1]); //second column of row
      var yearOfEvent = eventDate.getFullYear();
      var monthOfEvent = eventDate.getMonth() + 1;
      var dayOfEvent = eventDate.getDate();
      if (currentYear >= yearOfEvent && (currentMonth > monthOfEvent || (currentMonth == monthOfEvent 
           && currentDay >= dayOfEvent)))
      {
        var subject = emailSubjectPrefix + row[2];  //third column of row
        var message = row[3]; // fourth column of row
        message = "\r\n\r\n" + message + "\r\n\r\n";
        //Add a link to the spreadsheet in the email so people 
        //can easily go disable the reminder 
        message = message + "\r\nSent on " + currentDate + 
        "\r\nDisable the notification by changing the date on it here: "
        + urlToGoogleSheet;
        message = message + "\r\nReminder Start Date: " + eventDate
        MailApp.sendEmail(emailAddress, subject, message);
      }
    }
  }
}

Posted in Code Design, CodeProject, Uncategorized | Leave a Comment »

How to Upgrade to a Stronger Password Hash. Such as Upgrading from MD5 to BCrypt.

Posted by robkraft on July 23, 2018

Years ago I upgraded the hash algorithm in our database application from a custom algorithm to BCrypt.  Although the implementation went well I lamented the need to retain the old algorithm until such time as all users had logged in and replaced their old hashed password with a new hashed password.  Since that time I have learned there is a better way to upgrade to a stronger password hash so I am sharing it here in case I need to do this again in the future and possibly to help someone else implement a better approach.

In a nutshell, what I should have done is hashed all the existing hashed passwords using BCrypt immediately.

If that does not give you the hint you need to see the solution, allow me to explain in more detail.

Assume you have a database full of user names and passwords, the passwords were all hashed using MD5, and you now want to use BCrypt to hash them.  The approach a lot of us take, including myself, is to add the BCrypt algorithm to the application code.  When users log in to the updated application, the code does the following:

  • Person enters user name and password
  • Application hashes the password using MD5
  • If the MD5 hash matches the hashed MD5 password in the database:
    • The application hashes the password using BCrypt and stores that in the database
  • In the future, the application will use the BCrypt hash for the user instead of the MD5 password.

The problem with this approach, is that that the application must continue to support the old hash algorithm (MD5 in this example), until all users have logged in and converted their password hashes to BCrypt.

There is a better way.

  • Immediately hash all the MD5 hashes using BCrypt.  (Note, this is not hashing the passwords, because we do not know the passwords.  We are hashing the MD5 hash of the passwords.)
    • This allows us to immediately have all the passwords hashed with the stronger BCrypt algorithm.
  • As users log in to the application, hash the password they enter using BCrypt and see if it matches the hash stored in the database.
    • If not, then hash the password they enter using MD5 (your old algorithm), and then hash the result of that operation using BCrypt.  If that hash matches what is stored in the database, allow the user access to the application and also create a new hash of their password directly using BCrypt and update the hash stored in the database.

As mentioned, the benefit of this approach is that you immediately convert all the stored passwords to the stronger algorithm, instead of implementing a gradual process that does not convert all the passwords until all user accounts have eventually logged in, which could be never for some applications.

Posted in Code Design, Security, Uncategorized | Leave a Comment »

Robert’s Rules of Coders: #11 Separate User Interface Logic From Business Logic

Posted by robkraft on July 10, 2016

One goal to keep in mind as you write software is to write software that is easy to maintain and enhance in the future. We can do this by organizing code so that things that might change will be easier to change. Consider this example:

CodeArt12-1

CodeArt12-2

In the code above, User Interface (UI) code is mixed together with the business object code. We should try not to pass details about how the UI implements a feature unless the business object really needs to know. In this example, the Products business object really only needs to know three pieces of information from the UI:

  • The Price
  • Should a discount be calculated because this is for a nonprofit agency? (yes or no)
  • Should a discount be calculated because this is a bulk purchase? (yes or no)

If we change the code to pass boolean values to the Products business object instead of the checkboxes, we gain the following benefits:

  • The UI can easily be changed in the future to use something other than checkboxes, and this change will not require also changing code in the Products business object.
  • We increase our potential to use the Products business object with different types of user interfaces. This business object may currently expect a C# WPF checkbox control, which means the business object would not work if someone had a C# Windows checkbox control, or perhaps a C# Silverlight checkbox control. But if the Products business object accepted a boolean, which is a datatype common to more platforms, the business object will more likely work with different user interfaces.
  • Unit tests that we write won’t need to reference the specific UI components needed for building the user interface.

A more common way that developers often entwine UI code with business object code is shown below. This example is the opposite of the case above. Here logic that could, and should, reside in the business object is performed in the UI.

CodeArt12-3

CodeArt12-4

The reason we don’t like this code is that logic to calculate the discounted price should be moved from the UI to the Product business object. In doing so we would gain the following benefits:

  • We could reuse the Product business object in another program without needing to also add logic to the UI of the other program to calculate the discounted price.
  • If we need to change the calculation for the discounted price, we need to make the change in only one place and every program using that business object automatically is affected.
  • We can easily write a unit test on the Product business object to make sure that the code calculating our discounted price works correctly.

A better way to write the code from both examples above so that the UI and business logic is not entwined is shown below. I will admit that this is not the best example, because it does not use TryParse, nor does it have input checking and error handling, and it could use an interface, but those topics are not the point of this article, which is to encourage you to separate the UI logic from the business logic in your applications.

Codeart12-5

CodeArt12-6

It is not bad sometimes to write code that entangles UI code and business logic, knowing that you will refactor the code to move the logic to the correct place before you consider the feature complete. It is often helpful to have all of the code in one big method until you have it correct, then you can improve the code by refactoring it.

As with any of Robert’s Rules of Coding, you don’t need to adhere to them all of the time and there are cases where it is better not to. But most programmers should follow the rules most of the time. I hope you agree.

Go to Robert’s Rules of Coders for more.

 

 

Posted in Code Design, CodeProject, Coding, Uncategorized | 1 Comment »

Robert’s Rules of Coders: #10 Scope minimally, scope private

Posted by robkraft on May 22, 2016

A big ball of mud is a designation for some software programs that are very difficult to maintain. We never set out to create a big ball of mud, but it happens to many programs over time. One technique to reduce the entanglements that lead to a big ball of mud is to expose the functions in your code as little as possible to other parts of your program. In many programming languages the term “scope” is used to determine if your function can be used anywhere within the program, or just within a small area of the code.

The two most common scopes are “public”, which usually means that any part of your program, and perhaps other programs, can call the function; and “private”, which usually means that only other functions within the same region of code, within the same “class” or “namespace”, or perhaps within the same compiled assembly can call the function.

Some languages include additional scopes such as “internal”. .Net allows methods that are scoped private to only be called by other methods within the same “class”; but the use of the “internal” scope allows other classes within the same compiled dll to call the methods with an internal scope, but code in other dlls cannot.

When you write a method that is only used within one class, you should always set the scope of that method to private. This provides the following benefits:

  1. You do not unintentionally provide access to sensitive data.
  2. The code will not be accidentally called from another class, by another developer or yourself, creating an unexpected and undesired entanglement.
  3. The code will not be called by third-party developers, particularly if the class you are writing is part of an API that you expose to third-party developers.
    1. Of course, if you desire that third-party developers have use of the method then scope it public, but do not do so by default because you may be unable to change the method in the future without breaking the third party dependencies on your API.
  4. If you have scoped the method privately, you can be sure that other code is not using the method if you later decide to refactor or rename the method.
    1. This is not entirely true for languages that support alternate methods of calling even private functions from outside of the functions class. For example, the .Net language provides a feature called reflection that can be used to call the private members of another class. Developers using the reflection technique usually understand the risks.
  5. If you have a method that could be accessed by another class, you should spend a few minutes reconsidering the design of your classes instead of quickly and simply making the class public.
    1. Perhaps you should create a new helper class; perhaps you should refactor the code in the method to make accessible just what is needed in the other class.

In the two examples below, the developer desires to use the Base64Decode method in another class.  In the first example the ExampleOfWhatNOTtoDo class is made public as are all of the methods.

CodeArt10-1

But it would be better to move the Base64Decode method out of the ExampleOfWhatNOTtoDo class, so that nothing is made public that does not need tobe public as shown below.

CodeArt10-2

When the language allows a level of scoping besides public and private, such as internal, you should generally consider changing the scope from private to internal if that is sufficient rather than from private to public.

I have two final tips for .Net developers.

Tip 1: You can scope methods internally, yet allow other applications that you write have access to them but using “Friend” assemblies as described here: https://msdn.microsoft.com/en-us/library/mt632254.aspx. The InternalsVisibleTo attribute can be provided to the class in code or to the entire dll in the AssemblyInfo.cs file.

Tip 2: You can use an interface for internal methods, thus making your methods accessible through an interface from other dlls in your solution. You just need to explicitly implement the interface as described here: https://msdn.microsoft.com/en-us/library/ms173157.aspx.

As with any of Robert’s Rules of Coding, you don’t need to adhere to them all of the time and there are cases where it is better not to. But most programmers should follow the rules most of the time. I hope you agree.

Go to Robert’s Rules of Coders for more.

 

Posted in Code Design, Robert's Rules of Coders, Uncategorized | Leave a Comment »

Robert’s Rules of Coders #9: Eliminate Deep Nesting by Using Switch Statements and Functions

Posted by robkraft on February 14, 2016

The ‘If’ statement is one of the fundamental coding constructs of almost every programming language. Along with the ‘If’ statement, most languages also support ‘else’ conditions and the ability to nest ‘If’ statements. But this simple construct can also become one of the biggest contributors to code that is difficult to understand and modify. This often happens when ‘If’ statements get nested within ‘If’ statements; but there are two simple techniques you can use to reduce this complexity, ‘Switch’ statements and functions.

Switch statements offer these benefits to most developers:

  • They are easier to read, and thus
  • They are easier to understand, and thus
  • They are easier to maintain.
  • They are also easier to debug
  • In many languages, they also can be compiled to execute a little more swiftly that nested ‘If’ statements

Here is an example of an ‘If’ statement than can be improved by converting it to a ‘Switch’ statement:

If aValue = 6 then

Stars = stars + 1

Else

if aValue = 7

Stars = stars + 3

Else

if aValue = 8

Stars = stars + 5

Else

if aValue = 9

Stars = stars + 9

End if

End if

End if

End if

Here is the same logic from above, using a ‘Switch’ statement:

Switch aValue

Case 6:

Stars = Stars + 1

Case 7:

Stars = Stars + 3

Case 8:

Stars = Stars + 5

Case 9:

Stars = Stars + 9

I suspect that you will agree that it is easier to understand the code in the switch statement than the code in the nested ‘If’s. Another technique to eliminate nested ‘If’ statements is to move some of the code into separate functions. Although the hierarchy of ‘If’ statements may remain the same from the computer’s point of view, to most humans it becomes much easier to manage.

If input data is valid

If filename is valid

Create File

If file was created

Log “Success”

Return “Success”

Else

If error due to size

Log “Failure”

Return “Could not create file because it is too large.”

If error due to permission

Log “Failure”

Return “Could not create file because you do not have permissions.”

Else

Log “Failure”

Return “Unable to create the file. Reason unknown.”

End if

End if

Else

Log “Failure”

Return “Your file name is invalid.”

End if

Else

Log “Failure”

Return “The file input is invalid.”

End if

Here is the same logic from above, using functions:

String response = “”

Response = IsInputValid(myinput)

If (response= “”)

Return response

Response = IsFileNameValid(myfile)

If (response= “”)

Return response

return FileCreationResultMessage(myfile, myinput)

The functions called from the code above:

Function string IsInputValid(string input)

If input is not valid

Log “Failure”

Return “The file input is invalid.”

Else

Return “”

End if

End Function

Function string IsFileNameValid(string input)

If input is not valid

Log “Failure”

Return “Your file name is invalid.”

Else

Return “”

End if

End Function

Function string FileCreationResultMessage(string file, string input)

Create File

If file was created

Log “Success”

Return “Success”

Else

If error due to size

Log “Failure”

Return “Could not create file because it is too large.”

If error due to permission

Log “Failure”

Return “Could not create file because you do not have permissions.”

Else

Log “Failure”

Return “Unable to create the file. Reason unknown.”

End if

End if

End Function

As with any of Robert’s Rules of Coding, you don’t need to adhere to them all of the time and there are cases where it is better not to. But most programmers should follow the rules most of the time. I hope you agree.

Go to Robert’s Rules of Coders for more.

Posted in Code Design, CodeProject, Coding, Robert's Rules of Coders | 6 Comments »

Robert’s Rules of Coders: #8 Avoid Negative Conditionals And Method Names

Posted by robkraft on January 20, 2016

When you write code you should almost always assume that another person will attempt to read and understand that code some day in the future. That person could be your future self. Therefore, it is in your best interest to write code that can be quickly and easily understood by people, in addition to providing the correct instructions to the computer. Two ways to do this are to avoid negative conditions and negative words in method and variable names.

Avoid Negative Conditions

Let’s consider the following two ways you could write a bit of logic:

You could write:

If a > b then… //If a is greater than b

Or you could write:

If a !<= b then.. //If a is not less than or equal to b

 

A computer will understand both ways equally well, but it usually takes humans a little longer to understand the second way than the first. And when another developer adds a little more logic to the statement it takes humans even longer to figure out as shown in the next two statements:

 

You could write:

If a > b and c > a then… //If a is greater than b and c is greater than a

 

Or you could write:

If a !< b and c !< a then… //If a is not less than or equal to b and c is not less than or equal to a

 

When developers see the second way above they will probably write down numbers on a piece of paper to help them deduce the logic, even though ultimately it is the same as the first way. Of course changing the sequence of the conditions can also make the logic more readable, but in some languages that may come with a performance impact.

 

Consider Performance Impact of Condition Sequence

You could write:

If a > b and c > a then …

 

As

If c > a and a > b then …

 

Both ways provide the same output, but many languages will only evaluate the second condition if the first condition is true. This happens because the language is smart enough to realize that if the first half of the statement is false then the entire statement will be false. So in some cases, like in the methods below, it is probably more efficient to write the code the first way than the second way even though the results are the same:

 

The first, more efficient way to write the code:

If FastMethodThatReturnsTrueHalfTheTime() == true and SlowMethodThatUsuallyReturnsFalse() == true then…

 

The second, less efficient way to write the code:

If SlowMethodThatUsuallyReturnsFalse() == true and FastMethodThatReturnsTrueHalfTheTime() == true then…

 

 

Avoid Negative Words in Method Names

We often include method names in our ‘if’ statements and therefore we should probably avoid negative terms in the names of our methods. Consider the following examples of a method that checks for hyphens in a string:

 

String streetName = “Happy-Road”

If StringContainsHyphen(streetName) Then …

If StringDoesNotContainHyphen(streetName) Then …

 

The two statements above both seem easy to understand, but what happens if a coder checks if the results are false? Notice that it gets more difficult to interpret the second statement that contains the negative word “Not” in the name.

 

If StringContainsHypen(streetName) == false Then …

If StringDoesNotContainHypen(streetName) == false Then …

 

Avoid Negative Words in Variable Names

The guidance to avoid negative words in method names holds similar value for variable names, particularly the names of boolean variables.

 

We could name a boolean blnCountExceedsMaximum, or blnCountDoesNotExceedMaximum. We could also name it blnCountIsBelowMaximum or blnCountIsNotBelowMaximum. Now look at the following four statements and decide for yourself which is easiest to understand.

 

  • If blnCountExceedsMaximum == false then …

  • If blnCountDoesNotExceedMaximum== false then …

  • If blnCountIsBelowMaximum == false then …

  • If blnCountIsNotBelowMaximum== false then …

 

If you are like me, the second and fourth statements take a little longer to understand than the first and third.

 

Should you include “Is True” and “Is False”?

I have one last topic I think you should consider when writing ‘if’ statements. Should you include the “== true” and “==false”, or just leave the variable without that? A lot of languages do not require those values. The computer handles the following two statements equally well.

 

If blnSuccess == true

 

If blnSuccess

 

Personally, I don’t think it usually matters too much and I tend to code both ways. Sometimes, usually when my if statement has a method call in it or multiple conditions in it, I will include the “== true” or “== false” because I believe it adds clarity:

 

If (ExportDataToFile() == true and LastExportSucceeded == false) then

 

If you are going to test if a non-boolean variable is true or false, then I recommend including the “== true” or “== false” to let the next programmer know that this is what you intended.

 

//You could test if “number of failures” is not equal zero this way

If NumberOfFailures == true

 

//Or you could test if “number of failures” is not equal zero this way

If NumberOfFailures

 

In the second statement, it is not obvious that you intended to test the number as a boolean. Another programmer looking at the code might think you accidentally left part of the code out of the program. Rather than use either of those statements, it would probably be best to write:

 

If NumberOfFailures != 0

 

As with any of Robert’s Rules of Coding, you don’t need to adhere to them all of the time and there are cases where it is better not to. But most programmers should follow the rules most of the time. I hope you agree.

Go to Robert’s Rules of Coders for more.

Posted in Code Design, CodeProject, Robert's Rules of Coders | Leave a Comment »