1. Data centers have existed for decades. Without them, we wouldn’t have the internet, smartphones, websites, Facebook, Netflix, Amazon, X, or even email. Most people know this in the abstract, but given how much animosity has been directed at data centers over the past year or two, it’s worth saying plainly: tens of thousands of data centers around the world quietly power virtually everything we do “online.” A world without them isn’t a simpler or greener world — it’s a world without modern banking, hospitals, logistics, or communication. Whatever the right policy is, “no more data centers” isn’t it.
2. The concerns driving the backlash are not unreasonable, and pretending otherwise won’t win anyone over. Data centers use significant electricity and, depending on their cooling design, significant water. They can strain local grids, raise utility rates for nearby residents, generate low-frequency noise, and receive tax abatements that don’t always deliver the promised local jobs. Communities that feel they’re absorbing the costs while the benefits flow elsewhere have a legitimate grievance. The right response to those concerns is better siting, better cooling technology (air-cooled and closed-loop systems use far less water), honest negotiation over tax incentives, and grid investments paid for by the operators who drive the demand. The wrong response is to treat every proposed data center as inherently illegitimate.
3. We also need to keep building, because demand for computing is growing — and artificial intelligence has accelerated that demand sharply. AI may turn out to be the most significant technological shift since we harnessed electricity. That’s a prediction, not a fact, and reasonable people disagree about the magnitude. But even on conservative estimates, AI is already reshaping how knowledge work gets done, and the compute it requires has to physically exist somewhere.
4. Here’s the part that gets less attention in the debate: people with access to capable AI are going to be significantly more productive than people without it, especially for digital work. AI is expensive today because demand is strong, supply is limited, and energy costs are high. Wealthy people and large companies can comfortably afford current prices — and could still afford prices ten or a hundred times higher. If supply stays constrained, the productivity gains from AI will flow disproportionately to those who can already afford them, widening the gap between the wealthy and everyone else.
5. The only way to put AI within reach of ordinary people is to bring costs down, and the only way markets accomplish that is by expanding capacity. That means more data centers. The encouraging news is that the per-unit cost of AI has already been falling sharply as capacity has grown — evidence that the mechanism works when we let it.
Conclusion. None of this means every data center on a drawing board should be built. Given the pace of announcements, it’s entirely plausible that too many will be built too quickly, wasting capital, land, water, and power. Local communities deserve real input, real protections, and a real share of the benefits. But the opposite extreme — blocking new construction across the board — carries its own danger: that the productivity gains from AI flow only to those wealthy enough to afford scarce, expensive compute, while everyone else falls further behind. The honest debate isn’t “data centers, yes or no.” It’s where to build them, how to power them, who pays the costs, and who shares the gains. That’s the conversation worth having.
I was surprised by how difficult it was to set up a MySQL database on AWS RDS to use IAM Authentication. IAM Authentication is superior to using connection strings because you don’t need a password to connect, therefore you don’t need to find a way to securely store and manage that password.
Step 1: Configure RDS database for IAM Authentication
To set up your MySQL installation on AWS RDS you first need to configure your MySQL database in RDS to allow IAM Authentication.
At the time of this writing in September of 2024, you do that by navigating to your database in AWS RDS and selecting “Modify”. In the Database authentication section, select “Password and IAM database authentication”
After doing so, you need to click the “Continue” button then “Modify DB Instance”. The change does not happen immediately. When I did this, I had to repeat the process several times because my change did not seem to take. So you should go back into your database configuration to confirm the change took. Also, I believe the database needs to be rebooted before it takes effect. I recommend that you wait to move on to subsequent steps until you have confirmed, via the Modify UI for the database, that the Database authentication value is set correctly.
Step 2: Create a policy that allows connecting to the database
This step was the most problematic for me and took the longest to figure out. I tried giving my Role all RDS policies I could find, but the only way I could solve the problem was to create a new policy. This is extremely bizarre and I still feel I must have overlooked something. If you know what I missed, please let me know, but this step is the final action I took that allowed my application to work.
Create a new policy (because there is not one built into AWS) that allows for “connecting” to an RDS database.
In IAM, go to Policies, select “Create policy”, select a service which must be “RDS IAM Authentication”. Select the checkbox for “All RDS IAM Authentication actions” (there is only one). Select “Any in this account” and “Specific” under Resources specify which AWS Resources can use this policy. Give your policy a meaningful name such as “CustomPolicyForConnectingToRDS” and save it.
Step 3 Create a Role in AWS for the policy and services
The great thing, in my opinion, about building applications in AWS or Azure is the ability to security them easily with policies. In this case, you need to now create a role that will have the new policy. In AWS IAM, create a new role or alter an existing role to contain all the policies that your service calling RDS will need. In my case, I will have several Lambdas that need to connect to my RDS MySQL database, so I created a role named LambdasToCallRDS and gave the role two permissions, the permission I created in Step 2 “CustomPolicyForConnectingToRDS” and also “AmazonS3ObjectLambdaExecutionPolicy”.
Step 4: Assign the role to your services
The last thing you need to do in AWS is to configure your service, in my case a Lambda, to have the role you created in Step 3. So I went to the configuration of my Lambda and to the Permissions tab and assigned my role to the Lambda.
Step 5: Create user in MySQL
You need to create a user that logs in using AWS’s IAM. I don’t think you can pick an existing user in MySQL that uses a login and password, but I am not sure about that. I created a new user in RDS as shown here with the “IDENTIFIED WITH AWSAuthenticationPlugin” and limited the permissions of that user to the CRUD verbs.
CREATE USER ‘myrdsuser’@‘%’ IDENTIFIED WITH AWSAuthenticationPlugin as ‘RDS’;
GRANT SELECT, INSERT, UPDATE, DELETE ON reusefull.* TO ‘myrdsuser’@‘%’;
Step 6: Write some code to use IAM Authentication
The _dbUser is the name of the user you created in MySQL. In my example above, the name is myrdsuser.
I gave SQL Server’s “backup to Azure” feature a try. The feature rolled out in SQL Server 2022. It was fairly easy to set up, but it wasn’t for me, and it created a lot of traffic (SQL Queries) to my SQL Server. So I decided to turn it off. But that is incredibly hard. It is now a year later and I think I may have succeeded.
I started by trying to turn off services on my local PC related to it, but there was still SQL Traffic and errors in Event Viewer. Over the months, when I had time, I uninstalled SQL Server features using Add/Remove programs, but the problems kept coming. I restarted SQL Server after doing these things. I used resources at https://learn.microsoft.com/en-us/azure/backup/backup-sql-server-azure-troubleshoot to help identify things I might uninstall, delete, remove.
Finally today I found a task in Windows Task Scheduler, under Microsoft and under SqlServerExtension that I think is the culprit behind my woes. I deleted it, and am watching my logs, SQL Profiler traffic and hoping to see no more SQL Server Extension traffic or errors.
Microsoft Access databases downloaded from a website or received via an email can often be deployed in less than a minute. But if you don’t do it often, you may be unaware of security features in Windows that may require a few extra steps. In short, the steps are 1) Unzip, 2) Unblock, 3) Confirm opening, and 4) Enable Content
Unzip the file.
If the file is zipped, you must first unzip it. Zipping files before sending them in email is common because it makes the files smaller, and sometimes helps keep the email servers from flagging the Access database attached to the email as possibly malicious.
Unblock the file.
Windows automatically “blocks” some files when they are placed on your hard drive after being downloaded from the Internet or received via email. The block is intended to prevent malware from automatically executing and damaging your computer.
Confirm opening the file.
The first time you open a file that may trigger running code, Windows is likely to prompt you to confirm that you want to do so. Just click the Open button to do so.
Open the database and Enable Content.
Like most products in the Microsoft Office Suite, Microsoft Access has a feature called Macros that exists to run code. Most Microsoft Access applications rely on Macros in order to work properly, though some do not. It is very likely that you need to click on “Enable Macros” to use your Access database.
That is likely all that you need to do for most Access Databases.
If you open the Access Database and see the “Security Risk Microsoft has blocked macros from running because the source of this file is untrusted.” error message shown below, go back to the properties of the file and make sure that it has been successfully “unblocked”.
While it’s important to execute projects well, the true measure of success lies in the quality, impact, and longevity of the product, not the completion of the projects that built the product. Sacrifice a project for the good of the product.
If you have been working on a software development project that you expect to take three years to complete, and you learn something new along the way, or a new technology like chap-gpt comes along that gives you new ideas, don’t be afraid to cancel the project and call the project a failure. The product is more important than the projects used to build it. Always do what is best for the product, even if it means a project is delayed or cancelled.
I’ll admit this post would have been more valuable fifteen years ago, but those still using the ASP.Net Membership provider tables and stored procs that came with .Net Framework 2.0, 3.0, or 4.8 my like the performance boost you can get from these changes. The Membership tables were designed with two major flaws, in hindsight. One, the primary keys are guids, which provide terrible performance, and two, they assumed most installations would have multiple applications using the same membership table, which I think they do not. But Microsoft made the application ID, a guid, part of every key in every query. If your application only has a single application ID, you can change the stored procs to not consider the application ID as part of the query, and change the indexes to remove application ID as well. The change scripts are provided below:
alter PROCEDURE [dbo].[aspnet_Membership_GetUserByEmail]
@ApplicationName nvarchar(256),
@Email nvarchar(256)
AS
BEGIN
IF( @Email IS NULL )
SELECT u.UserName
FROM dbo.aspnet_Users u, dbo.aspnet_Membership m
WHERE
u.UserId = m.UserId AND
m.LoweredEmail IS NULL
ELSE
SELECT u.UserName
FROM dbo.aspnet_Users u, dbo.aspnet_Membership m
WHERE
u.UserId = m.UserId AND
@Email = m.LoweredEmail
IF (@@rowcount = 0)
RETURN(1)
RETURN(0)
END
GO
alter PROCEDURE dbo.aspnet_Membership_FindUsersByEmail
@ApplicationName nvarchar(256),
@EmailToMatch nvarchar(256),
@PageIndex int,
@PageSize int
AS
BEGIN
-- Set the page bounds
DECLARE @PageLowerBound int
DECLARE @PageUpperBound int
DECLARE @TotalRecords int
SET @PageLowerBound = @PageSize * @PageIndex
SET @PageUpperBound = @PageSize - 1 + @PageLowerBound
-- Create a temp table TO store the select results
CREATE TABLE #PageIndexForUsers
(
IndexId int IDENTITY (0, 1) NOT NULL,
UserId uniqueidentifier
)
-- Insert into our temp table
IF( @EmailToMatch IS NULL )
INSERT INTO #PageIndexForUsers (UserId)
SELECT u.UserId
FROM dbo.aspnet_Users u, dbo.aspnet_Membership m
WHERE m.UserId = u.UserId AND m.Email IS NULL
ORDER BY m.LoweredEmail
ELSE
INSERT INTO #PageIndexForUsers (UserId)
SELECT u.UserId
FROM dbo.aspnet_Users u, dbo.aspnet_Membership m
WHERE m.UserId = u.UserId AND m.LoweredEmail LIKE @EmailToMatch
ORDER BY m.LoweredEmail
SELECT u.UserName, m.Email, m.PasswordQuestion, m.Comment, m.IsApproved,
m.CreateDate,
m.LastLoginDate,
u.LastActivityDate,
m.LastPasswordChangedDate,
u.UserId, m.IsLockedOut,
m.LastLockoutDate
FROM dbo.aspnet_Membership m, dbo.aspnet_Users u, #PageIndexForUsers p
WHERE u.UserId = p.UserId AND u.UserId = m.UserId AND
p.IndexId >= @PageLowerBound AND p.IndexId <= @PageUpperBound
ORDER BY m.LoweredEmail
SELECT @TotalRecords = COUNT(*)
FROM #PageIndexForUsers
RETURN @TotalRecords
END
go
ALTER PROCEDURE [dbo].[aspnet_Membership_FindUsersByName]
@ApplicationName nvarchar(256),
@UserNameToMatch nvarchar(256),
@PageIndex int,
@PageSize int
AS
BEGIN
-- Set the page bounds
DECLARE @PageLowerBound int
DECLARE @PageUpperBound int
DECLARE @TotalRecords int
SET @PageLowerBound = @PageSize * @PageIndex
SET @PageUpperBound = @PageSize - 1 + @PageLowerBound
-- Create a temp table TO store the select results
CREATE TABLE #PageIndexForUsers
(
IndexId int IDENTITY (0, 1) NOT NULL,
UserId uniqueidentifier
)
-- Insert into our temp table
INSERT INTO #PageIndexForUsers (UserId)
SELECT u.UserId
FROM dbo.aspnet_Users u, dbo.aspnet_Membership m
WHERE m.UserId = u.UserId AND u.LoweredUserName LIKE @UserNameToMatch
ORDER BY u.UserName
SELECT u.UserName, m.Email, m.PasswordQuestion, m.Comment, m.IsApproved,
m.CreateDate,
m.LastLoginDate,
u.LastActivityDate,
m.LastPasswordChangedDate,
u.UserId, m.IsLockedOut,
m.LastLockoutDate
FROM dbo.aspnet_Membership m, dbo.aspnet_Users u, #PageIndexForUsers p
WHERE u.UserId = p.UserId AND u.UserId = m.UserId AND
p.IndexId >= @PageLowerBound AND p.IndexId <= @PageUpperBound
ORDER BY u.UserName
SELECT @TotalRecords = COUNT(*)
FROM #PageIndexForUsers
RETURN @TotalRecords
END
GO
ALTER PROCEDURE [dbo].[aspnet_Membership_GetUserByName]
@ApplicationName nvarchar(256),
@UserName nvarchar(256),
@CurrentTimeUtc datetime,
@UpdateLastActivity bit = 0
AS
BEGIN
DECLARE @UserId uniqueidentifier
IF (@UpdateLastActivity = 1)
BEGIN
-- select user ID from aspnet_users table
SELECT TOP 1 @UserId = u.UserId
FROM dbo.aspnet_Users u, dbo.aspnet_Membership m
WHERE
@UserName = u.LoweredUserName AND u.UserId = m.UserId
IF (@@ROWCOUNT = 0) -- Username not found
RETURN -1
UPDATE dbo.aspnet_Users
SET LastActivityDate = @CurrentTimeUtc
WHERE @UserId = UserId
SELECT m.Email, m.PasswordQuestion, m.Comment, m.IsApproved,
m.CreateDate, m.LastLoginDate, u.LastActivityDate, m.LastPasswordChangedDate,
u.UserId, m.IsLockedOut, m.LastLockoutDate
FROM dbo.aspnet_Users u, dbo.aspnet_Membership m
WHERE @UserId = u.UserId AND u.UserId = m.UserId
END
ELSE
BEGIN
SELECT TOP 1 m.Email, m.PasswordQuestion, m.Comment, m.IsApproved,
m.CreateDate, m.LastLoginDate, u.LastActivityDate, m.LastPasswordChangedDate,
u.UserId, m.IsLockedOut,m.LastLockoutDate
FROM dbo.aspnet_Users u, dbo.aspnet_Membership m
WHERE
@UserName = u.LoweredUserName AND u.UserId = m.UserId
IF (@@ROWCOUNT = 0) -- Username not found
RETURN -1
END
RETURN 0
END
GO
alter PROCEDURE [dbo].[aspnet_Roles_RoleExists]
@ApplicationName nvarchar(256),
@RoleName nvarchar(256)
AS
BEGIN
IF (EXISTS (SELECT RoleName FROM dbo.aspnet_Roles WHERE @RoleName = LoweredRoleName ))
RETURN(1)
ELSE
RETURN(0)
END
GO
drop index aspnet_Membership_index on [dbo].[aspnet_Membership]
go
CREATE CLUSTERED INDEX [aspnet_Membership_index] ON [dbo].[aspnet_Membership]
(
[LoweredEmail] ASC
)WITH (FillFactor=95) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [IX_ApplicationId] ON [dbo].[aspnet_Membership]
(
[ApplicationId] ASC
)WITH (FillFactor=95)
GO
drop index aspnet_Users_index on [dbo].[aspnet_Users]
go
CREATE UNIQUE CLUSTERED INDEX [aspnet_Users_Index] ON [dbo].[aspnet_Users]
(
[LoweredUserName] ASC
)WITH (FillFactor=95)
GO
drop index aspnet_Users_index2 on [dbo].[aspnet_Users]
go
CREATE NONCLUSTERED INDEX [aspnet_Users_Index2] ON [dbo].[aspnet_Users]
(
[LastActivityDate] ASC
)WITH (FillFactor=95)
CREATE NONCLUSTERED INDEX [IX_AspNet_Users_ApplicationID] ON [dbo].[aspnet_Users]
(
[ApplicationId] ASC
)WITH (FillFactor=95)
GO
Text Boxes on Forms and Reports in Microsoft Access allow you to configure the field to display “Rich Text” instead of “Plain Text” by setting the “Text Format” property of the control. This allows you to use HTML formatting in the field and display portions of the field using bold, italics, underline, and other enhancements.
However, when your data is coming from a SQL Server (or other DBMS), through linked tables, you can only choose to configure the control to support “Rich Text” when the field has a size of 256 bytes or greater.
Therefore, you need to go back to your source database (SQL Server), increase the size of the field so that it is 256 or larger, then go back to Access and Relink the table. After that, you should be able to choose Rich Text for the field on the form.
When the linked field length is too short, you will get this message from Microsoft Access “The setting you entered isn’t valid for this property.” when you try to change the control.
I learned some information I thought could help others more quickly resolve this error on SQL Server. As security demands increase, some features of older versions of products get more difficult to maintain. In this case, I am supporting a SQL Server running on SQL Server 2014. It has the latest Cumulative Update applied (CU) yet the emails sent via DatabaseMail are still failing. They were failing occasionally but began to fail consistently after applying one of those recent updates. In conjunction with the updates we applied registry entry changes in an attempt to get SQL Server to support TLS 1.2+ with Microsoft Office Email on Smtp.office365.com port 587.
The error we received from the failed emails was very vague:
The mail could not be sent to the recipients because of the mail server failure. (Sending Mail using Account 3 (2022-01-31T06:18:01). Exception Message: Cannot send mails to mail server. (Failure sending mail.).
I used this simple script to test sending emails:
declare @rc int
exec @rc = msdb.dbo.sp_send_dbmail
@profile_name = N'SQLSend',
@recipients = 'myemail@mail.com',
@subject = N'*** Test by me ***',
@body = 'email body'
I wrote a C# .Net program to see if I could get more error detail. I didn’t know what version of .Net Framework was on the server so I compiled against .Net 2.0 and ran my program and got the same error. Then I compiled my program on .Net 4.8 and got a different error. What I learned was that the newer version of the .Net framework (anything 4.0 or higher) provides a more detailed error message.
So my next task was to figure out if SQL Server is using the .Net Framework to send the Email (which I think it is) and how to tell SQL Server to use a newer version of the .Net Framework.
SQL Server uses an executable named DatabaseMail.exe to send the email. That program will exist in a folder named something like: D:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\Binn
By telling DatabaseMail.exe to use .Net Framework 4.5, we were able to get a more detailed error message from the failed emails. In our case, the error we got by using .Net Framework 4.5 is shown below. Our belief is that one of the SQL Server updates somehow altered the existing configured password of the hashing or encryption algorithm used for it. We simply re-entered our password in DatabaseMail configuration and our email started working again.
The mail could not be sent to the recipients because of the mail server failure. (Sending Mail using Account 3 (2022-02-01T12:47:47). Exception Message: Cannot send mails to mail server. (The SMTP server requires a secure connection or the client was not authenticated. The server response was: 5.7.57 Client not authenticated to send mail. Error: 535 5.7.139 Authentication unsuccessful, the user credentials were incorrect.
When the product owner provides a well-groomed backlog, you may think selecting the next feature to work on is obvious, but often it isn’t. A development team should consider many factors before picking an item from the product backlog to work on.
Here are some examples of how features may be selected for development:
Product Owner in Charge. In some contexts, [what is a context?], the development team is treated as an assembly line to deliver features, and the development team has little or no input into which features they will work on. A drawback of this context is that the development team isn’t allowed to provide an alternative sequence of features that could minimize development costs and increase throughput and quality. Additionally, allowing the development team to have input on the decisions can increase their commitment to completing the features as well as strengthen their “buy-in” to the product and process, which is known to increase morale and quality.
PMO in Charge. In some contexts, the development team supports several products and may work on features for a different product each iteration. In contexts where the development team has some input into feature selection decisions, it is often beneficial to the team to establish a Product Management Office (PMO) that works with each Product Owner to get a consensus agreement from the Product Owners choosing which feature, which Product Owner’s product, will get some development time during the next cycle. When the delivery team works on feature requests directly from customers, the team may find it helpful to establish a customer advisory board (CAB) that receives most of the large requests and provides input and direction to the development team about which features will be most valuable for most clients.
Too Many in Charge. In some contexts, development teams switch from feature to feature as their bosses, managers, and clients demand they stop working on one feature and switch to another to satisfy the demands from a specific customer or the whims of a forceful product owner or manager. Many drawbacks and inefficiencies exist in such a context. Developers are often demoralized by the inability to finish what they start so output is below expectations and quality is poor. Context switching wastes time as developers need to re-acquaint themselves with something new and lose the design ideas that existed only in their heads for the features they had been working on.
Waterfall. In non-agile environments, new projects and feature requests may get delivered to the development team(s) with the expectation that an expected delivery date for the feature will be provided soon even though none of the Product Owners have an idea of the current backlog for the delivery team. Equally unfortunate is the fact that the delivery teams may stop the feature development on other products when the new request comes in because they are more interested in learning what the new feature is, and they will spend time doing some design and development on it to return an estimate of the expected delivery date.
No One in Charge. In contexts without delivery expectations or engaged product owners, developers may drift into spending a lot of time developing features that are very unlikely to ever be used, or in developing features completely different than what the product owners expected. Additionally, developers may context switch often, working on one feature for a while and then another, leaving them with much started and nothing completed when the product owner eventually asks to see some results.
Customer in Charge.Some contexts have a particularly demanding or upset client that seems to be dictating the features a team will deliver. I once worked on a team where we obtained a new client (I will call them ACME), our largest client ever by far, and they soon demanded software improvements to help them meet their needs. So over the course of two years most of the features we delivered were those requested by ACME. We even joked that our developing methodology was ADD (ACME driven development) as opposed to Business Driven Development (BDD) or Domain Driven Development (DDD).
Development Team in Charge.Finally, in contexts where the team wants to work on the features with the greatest return on investment (ROI), developers often collaborate with product owners to calculate the ROI of each feature, providing a value for the feature from the perspective of those who will be using the software, and the developers providing an estimate of the difficulty to deliver the feature and the risks involved. Minimal risk, easy to develop, high value features are usually developed first; and high risk, difficult to develop features are delivered later even if their value is high. The textbook formula for ROI is value divided by development cost; and those with the highest ROI scores should be worked on earliest.
The Optimal Context
The optimal context for obtaining the best ROI from software development is a context in which the development team wields the majority of the power to decide the sequence in which features will be developed. The reason for this is the development team has the best knowledge of the changes needed to implement a feature and can suggest the best sequence of feature delivery to minimize development costs. In a healthy context, the development team will desire input from the product owners to identify the features with the highest value and attempt to deliver those first, and the product team will understand and usually be happy with the decisions made by the development team.
However, there are some contexts in which someone other than the development team can make better decisions about what to develop next. One context is during chaos caused by bad software that needs to be fixed immediately or important clients and customers threatening to leave if their demands are not met quickly. Another context is when the team is comprised of junior developers or people new to a product. In this latter context, it may be best to allow the product owner to determine the sequence in which the team will work on features until the team has learned the software architecture, development processes, and product domain.
Return On Investment (ROI) for Feature Selection
Although several of the contexts listed above may include the ROI (value/cost) when assigning the priority of feature development, the last context, Developer in Charge, is usually the context that provides the most accurate ROI calculation. The reason for this is that both the value and cost of a feature not yet developed change over time, but, with exception of time-sensitive features, the costs of a feature usually change more significantly than the value. The cost of developing a feature can be affected by many factors including:
The sequence in which features are delivered. In some cases, Feature A may be estimated to cost 100 hours to develop, and Feature B may be estimated to cost 40 hours to develop after Feature A. But if the sequence is reversed, and Feature B is developed first, B still costs 40 hours, but Feature A can re-use some of the code developed by Feature B and Feature A only costs 80 hours to develop.
The specific individuals assigned to do the work affect the cost, often substantially. A feature assigned to a junior developer available at the time may take 100 hours, whereas a senior developer may be able to deliver the same feature in 20 hours. Waiting for the senior developer to be available at the next iteration may be more efficient.
As developers write code and learn new technologies they sometimes discover a new approach to implement a feature that was previously not considered. The new approach may not only take less time, but also result in higher quality. This is one reason for delaying many technical decisions until you have more knowledge.
Hopefully, it is obvious that delivering the features with the highest ROI is most valuable to the company. If your organization is not using ROI as a factor in choosing what to work on next, then your team may be wasting time (as explained in Lean/Agile development philosophy), by not doing what is most valuable first. Allowing the development team to be the primary selector of the sequence in which to develop features may require a change in your team and company culture, and how management hierarchies are organized; but it may be necessary to become more efficient.
Challenges of Calculating ROI
At this point you may accept the premise that choosing features to work on based on their ROI is so sensible that everyone should use this approach, but the reality of software development is much more complicated. First of all, determining the ROI of each feature is very difficult. Developers know that estimating the time to deliver a feature is difficult but assigning a dollar amount to the value of a feature delivered may be even more difficult. So not only are both the numerator and the denominator of our ROI calculation fuzzy numbers, there is also a lot of room for variance within those two values. What is the ROI for delivering an order entry screen without date pickers for the date fields versus an order entry screen with date pickers for the date fields? What is the ROI of delivering a grid full of data that lacks the ability to filter the data versus a grid full of data that can filter the data? What is the incremental development cost of adding each desired sub-feature to the feature? Can you first deliver a basic set of features and the later deliver the enhancements? All these options and variables make ROI calculations for a single feature very subjective.
Challenges for using ROI for Feature Selection
You need to take time to perform ROI calculations and record them with the feature in your tracking system.
You need to have the people that best know how to value the feature from the customer perspective available to provide an estimated feature value.
You need to have the people that can best estimate the cost and risks of delivering the feature available to spend time to calculate an estimate.
The value of an undelivered feature can change over time as market and user needs change or as other options become available that help customers and users resolve their business needs.
The effort required to deliver a feature can change over time as the skills and knowledge of the developers change, but also as the technologies and frameworks of the software evolve and improve.
The cost to deliver some features may be reduced when the feature is delivered along with other features. Imagine having three separate requests to add a checkbox for different purposes to the same user interface, and the value of the first checkbox is deemed very high, but the value of the other two checkboxes is not. Each has the same development cost to deliver. However, if all three are delivered together the development costs decrease significantly because all three database changes can be made at once, and all three business object changes can be made at once, and all three UI changes can be made at once. Therefore, given that the cost of implementing checkboxes two and three, when implemented along with checkbox one, is reduced significantly, the ROI of checkboxes two and three would increase and it would make sense to deliver them along with checkbox one.
Another reason to implement features with lower ROI is to use them as a test case for a new technology. Perhaps you know you want to start encrypting data for millions of transactions, a feature with a lot of risk if it is delivered incorrectly. Therefore, you choose a lower priority feature from the backlog that also needs to encrypt data, but only in a small number of use cases. You can implement the feature with the lower ROI first to work out the difficulties of implementing the new encryption technology before using it on a more important feature.
There are more reasons for choosing to work on features that don’t have the highest ROI, and some of those reasons include:
You have a developer or two that need to be assigned some work, but the developers don’t have the skills to tackle the features with the highest ROI.
You have a junior developer or intern, and you want them to comfortably work on some features that will help them best learn your software and development processes; or you want them to work on some features that won’t have a significant impact if the developer makes a mistake.
ROI includes a time component. Value may decrease over time. One feature may provide a greater ROI initially, but another feature may provide a greater ROI over a longer time period. If a feature is working poorly in Windows 8 but works perfectly on Windows 10, the longer you wait to deliver the feature the less value the feature will provide when finally delivered because more of your customers have migrated to Windows 10.
ROI value may increase over time. We might estimate the value of a feature to be worth $50,000, but it may be more accurate to say that the feature will increase our revenue by $1,000 per day, thus its value is increasing over time. So, when we calculate the value for ROI and compare the ROI for two features, we need to consider the timeframe of the ROI for each.
Developers often prefer to work on features that provide the best ROI in the long term, which can include rewriting some old, buggy, difficult to maintain software using current frameworks and technologies. But if customers are angry and demanding the bugs get fixed ASAP, it may be best to spend time fighting the current fires, knowing that the very code you are fixing will be thrown out and replaced within a few months. It feels like a waste, but you must survive the short term to exist in the long term. If that phrase doesn’t make sense, consider this analogy: Fighting global warming may be the most important challenge we face and where we should focus our resources, but if the largest nations in the world are on the brink of a nuclear war, then focusing resources on resolving that challenge in the short term becomes more important than fighting global warming; because if don’t deal with the short term problem, no one may survive to address the long term problem.
Should the Development Team Do Activities Other Than New Features?
Some readers may feel there is a lot to consider to effectively use ROI to drive feature delivery but using ROI to determine what a development team does next is only part of the equation. There are activities you may find more valuable to your development process to work on instead of developing new features. If you have adopted a Continuous Process Improvement mindset, then, by definition, you believe you should often spend time considering how you can improve your processes; and in this context how you can improve your software development processes. What if you could make a change to your software development process that would increase delivery speed of new features by 50%? I suspect you would want to implement that change. But that change is not a new feature. It is more likely a DevOps change or a significant refactoring of the code, or a process change such as adopting Kanban. Regardless of what the change is, the change is likely to require time from your developers to implement and adopt. That means that instead of developing new features, your developers are going to spend their time on other activities. We now realize that there are activities other than developing new features that your development team should consider spending time on. Specifically it means that your software product and delivery process will get a better ROI when the development team performs activities like improving DevOps, fixing bugs, refactoring code, training, writing unit tests, and writing developer documentation than when they are developing new features. This is not to say that these activities always, or even most of the time, provide a better ROI than delivering new features, but it does mean that when the decisions are being made about what the development team should do next, the decision makers should not be considering only new features to deliver.
Evaluating the ROI of Possible Development Team Activities
The following diagrams provide examples of a backlog of activities that a development team could perform. The backlog has four types of items: New Feature, Fix Bug, Refactor, and DevOps change. In the upper half of the first diagram, the team chooses to implement a New Feature that has a development cost of 2, and the value perceived by the customer receiving the feature is 12 (100 – 88).
In the lower half of the above diagram, the development team chooses to fix a bug. The bug has a cost of 3 and increases the customer’s perceived value of the software by 2 (102 – 100). For both developers and businesspeople, the examples above are easily understood.
The following examples attempt to convey the value of Refactoring, DevOps Changes, and Training using the same principles of ROI.
In the upper half of the diagram below, the development team chooses to refactor some of their existing code. This code is delivered to production, but the customer perceives no value in the change (102 – 102 = 0). From the perspective of the customer, the features of the software have not changed (although in some cases refactoring may include noticeable performance or security improvements). But what you should notice is that the cost of several items in the Backlog changed after the refactoring was implemented. The most common effect of a successful refactoring is that delivering future features becomes easier. Of course, as with ROI calculations, it is difficult to estimate the impact on cost that a refactoring will have on other features. Additionally, refactoring code to support a newer framework or technology may be beneficial to changes made a year or more in the future, but not particularly valuable to the other features in the immediate backlog.
The lower part of the above diagram begins to show the impact of a change in DevOps. For purposes of this article, a DevOps change refers to a change made to the processes and tools used to build, unit test, validate, and deliver software. A DevOps change could imply that the team took time to automate builds instead of performing manual builds, or to automate deployment to a test environment so the QA team could test the latest build more quickly and provide feedback more quickly. Regardless, the DevOps change does not (in the case) affect production, and there is no deployment. The customer does not gain any direct value from the DevOps change. But, like refactoring, the DevOps change has a positive impact on subsequent development as shown in the diagram below. In this example, the Cycle Time, the speed at which a feature goes from “started” to “delivered” was reduced for most features developed after the DevOps enhancement. By investing time to make this DevOps improvement, all future development can be delivered faster. In this specific, admittedly fictional, example, the time for a feature with a size Estimate of “8” has been reduced from 25 to 22, the time for a feature with a size estimate of “5” reduced from 12 to 10, the time for a feature with a size estimate of “3” from 5 to 4; while the estimate for a feature with a size estimate of just “2” remains at 2.
One additional activity included in the lower part of the above diagram is developer training. The impact of developers taking time for training, such as learning new design techniques, coding techniques, or development product features, may be to increase the speed at which developers can produce features. So, like refactoring and DevOps enhancements, a team may consider developer training to be the activity that it is most valuable for a development team to do next.
Doing Multiple Things At The Same Time
The entire article up to this point assumes that development teams do one thing at a time. In reality, teams with more than three people usually do several things at the same time. Most teams, even many Kanban teams, will pick several features, bugs, and other activities to work on over the course of a few weeks, or during the next iteration. One benefit of doing multiple activities is that some team members are usually always working on new features, which is what product owners and customers are primarily interested in. Working on multiple activities at the same time allows some feature progress to continue even as other team members spend months on DevOps or refactoring activities.
Recap
There is value in understanding who decides what the development team will do next.
There is value in recognizing that the development environment context strongly influences who makes that decision.
Return on Investment (ROI) should be a primary factor in determining what is done next.
Developers usually can provide the best ROI estimates.
ROI is a subjective measure.
Activities such as refactoring and DevOps improvements may provide a better ROI than developing new features.
Rob Kraft's Software Development Blog 