In software development, performance optimization is a crucial topic and one of the most effective ways to improve the performance is caching. However, caching is a double edged sword we need to very careful about when and how to use caching. So, we need to use right strategy in right place.
Caching can be implemented at different levels, including memory, network, and CDN (Content Delivery Network) levels. Each level has a unique purpose and is suitable for different use cases. In this post, we will focus on memory-level caching which is simply storing the data in the memory of the machine where the code is being run. This type of caching is extremely fast and it is ideal for data that changes infrequently and is accessed frequently.
Network-level caching isn’t always easy but can be very helpful when you have many API replicas distributed across different servers. It allows the use of a distributed cache so all instances work with the same cached data, ensuring consistency levels and enhancing performance. Perhaps another article is in order.
I will be detailing how these caching strategies are developed through C# .NET in this article. Using the IMemoryCache interface is one of the most common ways to achieve memory-level caching in .NET Core. A well-implemented caching strategy can significantly enhance the performance and scalability aspects of your .NET codebase. There are a few strategies that might pique your interest.
1. Absolute Expiration
This is very basic caching. It is simply fixing cached data to expire at a definite point in time. This is most helpful for information that undergoes updates at consistent intervals. Let’s say, you have data that changes every hour. You can decide on an absolute expiration of one hour. When this hour elapses, the cached data is evicted and gets invalid. Upon the next request, the system will need to re-cache the information and set it to expire after another hour has passed.
The preferred method of implementing in-memory caching in .NET Core is through the use of the IMemoryCache interface. Let me give you a more detailed example to demonstrate how you can implement absolute expiration.
First of all, ensure that the memory cache service is added to your services within the Startup.cs file— this can be done as an alternative approach. The typical way of achieving this is by adding it in the ConfigureServices method. Afterward, inject IMemoryCache into your controller:
public void ConfigureServices(IServiceCollection services)
{
// All other service configurations...
services.AddMemoryCache();
}Next, inject IMemoryCache into your controller:
public class MyApiController : ControllerBase
{
private readonly IMemoryCache _memoryCache;
public MyApiController(IMemoryCache memoryCache)
{
_memoryCache = memoryCache;
}
}Now, let’s implement a method in the controller that uses absolute expiration for caching. Let’s suppose you have a method that fetches some data from a database or an external service. And you want to cache this data for a fixed duration:
public MyDataType GetData()
{
var cacheKey = "MyDataKey";
if (!_memoryCache.TryGetValue(cacheKey, out MyDataType cachedData))
{
var freshData = FetchData();
var cacheEntryOptions = new MemoryCacheEntryOptions
{
// Let's set cache time for 1 hour
AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1)
};
_memoryCache.Set(cacheKey, freshData, cacheEntryOptions);
return freshData;
}
return cachedData
}Consider using MyDataType as an imaginary representation for the type of data you’re caching in this case. The GetData method first checks if the data exists in the cache based on the key; if not, it acquires fresh data and sets an absolute expiration to one hour after storing it in the cache with that particular key. On subsequent requests, if the data is found within the cache, it just returns this saved information.
Your data is safe in the cache for sixty minutes and only for that period after being added to the cache. On expiry of this time, it shall be checked out and killed; a new one will come next hour upon another request.
2. Sliding Expiration
Sliding expiration is similar to absolute expiration but it is not absolute. In other words, it doesn’t just sit there waits to be removed from the cache based on a countdown time. It dynamically resets its timer each time someone comes looking for it. Sliding expiration would be highly suitable for situations where specific items need to be available within cache memory consistently throughout different requests— regardless of how frequently they are accessed or how long they stay dormant before being accessed again. Is your application responsive enough? Is it using memory effectively? Well, let sliding expiration take care by having such data always at hand without any extra effort. Of course, it has some disadvantages as well. If few specific items start occupying the cache space because they keep renewing their stay indefinitely due to high frequency calls.
Sliding expiration is particularly useful in scenarios where data access patterns are irregular but having immediate access to frequently accessed data is crucial. For instance, in web applications managing user session data, sliding expiration can ensure that active session information remains in cache as long as the user is actively interacting with the application. Similarly, for platforms providing personalized content, such as recommendation systems or user-specific settings, sliding expiration ensures that the data specific to an active user’s preferences or actions is readily available, enhancing the user experience by providing quick access to relevant information.
As an example, see Absolute Expiration above and instead of AbsoluteExpirationRelativeToNow, use SlidingExpiration setting a TimeSpan as follows:
var cacheEntryOptions = new MemoryCacheEntryOptions
{
SlidingExpiration = TimeSpan.FromMinutes(30)
};3. Priority-based Eviction
Priority-based eviction in caching is a technique where items in the cache are assigned different priorities. When memory becomes overloaded, items with lower priorities are removed first. This strategy is particularly beneficial in scenarios where memory resources are limited, as it allows for more control over what stays in the cache during memory pressure. By prioritizing crucial data, applications can ensure that the most important information remains accessible, while less critical data is evicted to free up resources. The key advantage of this method is its ability to maintain system stability and performance by intelligently managing memory usage. However, the challenge is correctly assigning priorities since misjudging the importance of data can lead to performance issues if essential data is evicted prematurely.
Priority-based eviction in real-life scenarios has value when the memory size is a limitation or when the cost of memory use needs optimization. For instance, in a financial application where certain transactions or data queries are more critical than others, priority-based eviction can ensure that high-priority financial records or frequently accessed user data remain in cache under memory pressure. In a content delivery system for example, newer or more popular content can be given higher priority over older, ensuring that the most demanded content remains quickly accessible to users.
To implement priority-based eviction in a .NET Core Web API, you would typically use the MemoryCacheEntryOptions class to set the priority of cache entries. After injecting IMemoryCache into your controller, you can set the priority of each item when adding it to the cache. Here’s an example:
//... all DI and initiation stuff...
public void CacheData(string key, MyDataType data, CacheItemPriority priority)
{
var cacheEntryOptions = new MemoryCacheEntryOptions()
.SetPriority(priority);
_memoryCache.Set(key, data, cacheEntryOptions);
}In this code, CacheData represents a function where you can specify the cache key and the data that needs to be cached (MyDataType), along with the cache item’s priority. Priority is set by invoking the SetPriority method from the MemoryCacheEntryOptions object. There are different priority levels available like CacheItemPriority, high or low which allows you to express how important the cached item is. This configuration ensures that the cache is able to smartly handle memory usage— keeping high-priority items in memory longer under memory pressure. The main idea behind this approach is that it guarantees critical data remains available for as long as possible, allowing users easy access to important information even when resources are constrained.
4. Custom Eviction Policies
Custom eviction policies allow developers to define their own rules for how and when cache entries should be removed. They are tailored to the unique requirements of their application. This approach offers the flexibility to create sophisticated caching strategies that align with specific data access patterns and application behaviours. The main advantage of custom eviction is its ability to optimize cache performance and efficiency in complex scenarios where standard eviction policies might not be sufficient. They enable fine-grained control over cache behaviour, ensuring that caching aligns closely with application needs. However, the downside is the added complexity in implementation and the potential for errors. Developing custom eviction logic requires a deep understanding of both the application’s data usage patterns and the implications of different caching behaviours, which can be challenging and time-consuming.
Here’s a simplified example:
public class CustomCacheEvictor
{
private readonly IMemoryCache _memoryCache;
private readonly ConcurrentDictionary<string, CacheItemInfo> _cacheItemInfos;
public CustomCacheEvictor(IMemoryCache memoryCache)
{
_memoryCache = memoryCache;
_cacheItemInfos = new ConcurrentDictionary<string, CacheItemInfo>();
}
public void AddOrUpdateItem(string key, object value, CustomEvictionCriteria criteria)
{
var cacheEntryOptions = new MemoryCacheEntryOptions()
.RegisterPostEvictionCallback((k, v, reason, state) =>
{
_cacheItemInfos.TryRemove(k.ToString(), out _);
});
_memoryCache.Set(key, value, cacheEntryOptions);
_cacheItemInfos[key] = new CacheItemInfo { Key = key, Criteria = criteria };
}
public void EvaluateCacheItems()
{
foreach (var itemInfo in _cacheItemInfos.Values)
{
if (ShouldEvict(itemInfo.Criteria))
{
_memoryCache.Remove(itemInfo.Key);
}
}
}
private bool ShouldEvict(CustomEvictionCriteria criteria)
{
// Logic to determine if the item should be evicted
// based on the custom criteria
}
}
public class CacheItemInfo
{
public string Key { get; set; }
public CustomEvictionCriteria Criteria { get; set; }
}
public class CustomEvictionCriteria
{
// Define properties and methods for custom criteria
}In this example, CustomCacheEvictor is a service that manages cache entries and their eviction based on custom criteria encapsulated within CustomEvictionCriteria. The AddOrUpdateItem method adds items to the cache and tracks them with associated criteria, while EvaluateCacheItems iterates through the tracked items and evicts them based on the custom logic defined in ShouldEvict. This approach provides a high level of control and flexibility, allowing the cache to adapt to the specific requirements and dynamics of the application.
This example demonstrates the usage of CustomCacheEvictor as a service that takes care of cache entries and removing them by specific custom conditions held within CustomEvictionCriteria. When using the AddOrUpdateItem method, we add items to the cache and keep an eye on them with related criteria; later through EvaluateCacheItems, we go through these tracked items and evict based on our ShouldEvict’s custom logic. This approach results in a high level of control, not forgetting flexibility. With such features, caches are able to respond to their own unique requirements (as well as dynamics) belonging to any application.
5. Dependency-Based Caching
Dependency-based caching is a smart way to keep cache entries valid and it’s quite sophisticated in that it uses other elements (dependencies) like files, database records or even other cache entries to ensure consistency of the data held in caches with the actual data sources. The main strength of this approach is the automatic invalidation of cache entries when their dependencies change. This keeps the data up-to-date without compromising the integrity.
There are situations where dependency-based caching can be a good fit— particularly when it is essential that data consistency be maintained and the base data changes often or for no obvious reasons. Take for instance a content management system, the outcome of page development could be cached but must disappear if the content of the page changes. Likewise in e-commerce, product listings might easily slip into cache but should just as easily disappear if prices or stocks change. This approach guarantees that users always have access to the most recent information without unnecessary delays caused by frequently regenerating caches.
One method of implementing dependency-based caching in a .NET Core Web API involves using the ChangeToken class. This class helps create a token which upon specific conditions leads to an eviction from cache. Here’s a basic example:
public class MyApiController : ControllerBase
{
private readonly IMemoryCache _memoryCache;
private readonly IMyDependencyService _dependencyService;
public MyApiController(IMemoryCache memoryCache, IMyDependencyService dependencyService)
{
_memoryCache = memoryCache;
_dependencyService = dependencyService;
}
public IActionResult GetData()
{
var cacheKey = "MyDataKey";
if (!_memoryCache.TryGetValue(cacheKey, out MyDataType cachedData))
{
MyDataType freshData = FetchData();
var dependency = _dependencyService.GetDependency();
var cacheEntryOptions = new MemoryCacheEntryOptions();
cacheEntryOptions.AddExpirationToken(new CancellationChangeToken(dependency.Token));
_memoryCache.Set(cacheKey, freshData, cacheEntryOptions);
return Ok(freshData);
}
return Ok(cachedData);
}
private MyDataType FetchData()
{
// Logic to fetch data
}
}
public interface IMyDependencyService
{
CancellationChangeToken GetDependency();
}This is what the code contains: IMyDependencyService is a service that supplies a CancellationChangeToken. This token has a direct connection to a particular dependency, which may include but is not limited to, a database record or file instance. When any change occurs on the dependency itself\ for instance the file experiences any modification or the database record receives any update, then it results in cancellation of this token. The cancellation leads to invalidating any cache entry that was associated with this token. By these means, we make sure that our cache always represents what is currently present in the dependency; thus achieving consistency of data throughout our application’s different components.
6. Write Through and Write Behind Caching
Write-behind caching, on the other hand, initially writes data only to the cache, delaying the database write operation. This can significantly improve performance for write-intensive applications, as it minimizes the immediate cost of database IO operations. The trade-off, however, is the increased risk of data loss in the event of a system crash because the cached data may not have been persisted to the database yet. This approach also introduces complexity in terms of managing the synchronization between the cache and the database to ensure eventual consistency.
Write-through caching is particularly useful in applications where data integrity and consistency are critical, such as financial transaction systems or inventory management systems, where it’s essential that the database reflects the most current information. Write-behind caching is more suitable for applications where performance is a higher priority than immediate consistency, such as logging systems or user activity tracking, where the sheer volume of write operations can be a bottleneck.
Here’s a simplified example:
public class CacheRepository<T> where T : class
{
private readonly IMemoryCache _cache;
private readonly IDatabaseRepository<T> _databaseRepository;
public CacheRepository(IMemoryCache cache, IDatabaseRepository<T> databaseRepository)
{
_cache = cache;
_databaseRepository = databaseRepository;
}
public async Task SaveAsync(string key, T data)
{
// Write through: Save to cache and database simultaneously
_cache.Set(key, data);
await _databaseRepository.SaveAsync(data);
}
}
public interface IDatabaseRepository<T> where T : class
{
Task SaveAsync(T data);
}In this example, CacheRepository takes care of the management of the data. This approach used by the SaveAsync method is saving the data first in the memory cache, then followed by saving it to the database. This act ensures that the cache mirrors the most current information being stored, which conforms to the write-through caching model. For a write-behind caching implementation, it would be about saving the data solely to the cache at first and then writing them to the database in an asynchronous manner— perhaps in batches or after some delay period — all done with a view of enhancing performance optimization.
7. Region-based Caching
Region-based caching involves partitioning related data into different regions within the cache, which is where it gets its name. This approach helps if there is no single universal cache eviction policy, expiry policies are different for different regions. The major benefit of this technique is the ability to handle various types of data differently based on their usage patterns and importance levels. An example would be storing high access frequency data in a region that has a longer expiration time while less critical data can go into a region with strict eviction policies. This helps to use cache resources more efficiently, thereby enhancing application performance— but setting up a region-based cache is not straightforward due to its complexity. It calls for keen administration: improper configuration of regions or policies governing evictions may result in poor cache performance (or worse still) anomalies leading to data inconsistency.
Region-based caching has the upper hand when it comes to applications that deal with assorted data types differing in importance levels and access patterns. Let’s take an e-commerce application for instance: it would opt for distinct cache regions for product descriptions, user reviews, and pricing information with dissimilar caching strategies that echo their individual update frequency and user access pattern. In a similar fashion, a content management system could employ various regions for the storage of page content, user profiles, and system settings— enabling a customized cache management approach that acknowledges the unique essence of each data type
Here’s a simplified example:
public class CacheService
{
private readonly IMemoryCache _memoryCache;
public CacheService(IMemoryCache memoryCache)
{
_memoryCache = memoryCache;
}
public void CacheProductInfo(string productId, ProductInfo productInfo)
{
var cacheEntryOptions = new MemoryCacheEntryOptions()
.SetSlidingExpiration(TimeSpan.FromHours(6)) // Longer expiration for product info
.SetPriority(CacheItemPriority.High);
_memoryCache.Set($"ProductInfo_{productId}", productInfo, cacheEntryOptions);
}
public void CacheUserReviews(string productId, UserReview userReview)
{
var cacheEntryOptions = new MemoryCacheEntryOptions()
.SetSlidingExpiration(TimeSpan.FromMinutes(60)) // Shorter expiration for user reviews
.SetPriority(CacheItemPriority.Low);
_memoryCache.Set($"UserReview_{productId}", userReview, cacheEntryOptions);
}
}
public class ProductInfo { /* ... */ }
public class UserReview { /* ... */ }The CacheService class includes various methods to cache product information and user reviews differently based on the regions. These regions are identified by unique key patterns; for example, ProductInfo_{productId} and UserReview_{productId}). Each method implements specific caching policies due to different natures of the data— including distinct expiration times and priorities for cache retrieval. This design guarantees that each kind of data is handled according to its unique features and usage, which in turn ensures optimal use of cache resources leading to better application performance.
public class TaggedCache
{
private readonly IMemoryCache _memoryCache;
private readonly Dictionary<string, HashSet<string>> _tags;
public TaggedCache(IMemoryCache memoryCache)
{
_memoryCache = memoryCache;
_tags = new Dictionary<string, HashSet<string>>();
}
public void Set(string key, object value, IEnumerable<string> tags, TimeSpan? expiration = null)
{
var options = new MemoryCacheEntryOptions();
if (expiration.HasValue)
{
options.SetAbsoluteExpiration(expiration.Value);
}
_memoryCache.Set(key, value, options);
foreach (var tag in tags)
{
if (!_tags.ContainsKey(tag))
{
_tags[tag] = new HashSet<string>();
}
_tags[tag].Add(key);
}
}
public void InvalidateByTag(string tag)
{
if (_tags.TryGetValue(tag, out var keys))
{
foreach (var key in keys)
{
_memoryCache.Remove(key);
}
_tags.Remove(tag);
}
}
}In this TaggedCache class, there is a Set method that stores a cached entry along with tags related to it while InvalidateByTag invalidates all cache entries linked with a particular tag. This setup facilitates the management of cache entries that share common characteristics or dependencies, enabling more efficient and flexible cache operations tailored to specific application requirements.
Suleyman Cabir Ataman, PhD