@KLuuKer We're interested in learning more about your use case. Do you plan to use it with MessagePackSerializer? Anything else? This seems potentially useful if you need to slice byte arrays. Perhaps we could use this internally if/when we're using array pools.

Note: We'd have to ifdef the new API out of netstandard unless we make a whole new interface because netstandard does not support default interface methods, and we cannot make breaking changes to this interface.

Thanks for contacting us.

We're moving this issue to the .NET 8 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

Triage: optimizing this is likely going to help in some hot paths. We believe the API needs some design work though.

Note: We'll need to make sure netstandard2.0 works.

@halter73 yes I do use it with MessagePackSerializer (now with a extra array copy unfortunately), but I do write my own payloads first to the buffer before I read\write to it with messagepack (like version\compatibility flags etc).

Sometimes I also end up writing custom byte's rented from a ArrayPool that ends up returning a slightly larger buffer then my actual payload, and apparently I also use WebEncoders.Base64UrlEncode (decode) allot.

Can't wait until net8.0 so I can start using it
p.s. anybody have a working time machine handy?

Maybe something like a TryProtect \ TryUnprotect and people can input their own source and destination span's.
I understand the potential ways "normal" developers might be able to shoot themselves in the foot with it, if they actually put secret data back into the buffer pool before sanitizing the bytes first.

My team's services would benefit from this API as well. Our scenario involves StackExchange.Redis, where the values we want to encrypt are of type RedisValue (which can be implicit converted to ReadOnlyMemory<byte>).

https://github.com/StackExchange/StackExchange.Redis/blob/ae6419a164600b4b54dd7ce8a699efe8e98d8f1c/src/StackExchange.Redis/RedisValue.cs#L855

I think it would be sufficient for us to call ReadOnlyMemory<byte>.Span if an overload that accepted ReadOnlySpan<byte> was provided. Initially, I thought it would be nice if IDataProtector provided a method that accepted ReadOnlyMemory<byte>, but if it did, I think it'd need to call .Span internally to iterate/consume the contents of the array it's pointing at anyway, right?

To get the perf win, we'd have to avoid allocating the array in the API implementation as well, so we'd probably need/want to update these too:

Microsoft.AspNetCore.DataProtection.AuthenticatedEncryption.IAuthenticatedEncryptor.Decrypt(System.ArraySegment<byte> ciphertext, System.ArraySegment<byte> additionalAuthenticatedData) -> byte[]!
Microsoft.AspNetCore.DataProtection.AuthenticatedEncryption.IAuthenticatedEncryptor.Encrypt(System.ArraySegment<byte> plaintext, System.ArraySegment<byte> additionalAuthenticatedData) -> byte[]!
Microsoft.AspNetCore.DataProtection.ISecret.WriteSecretIntoBuffer(System.ArraySegment<byte> buffer) -> void
Microsoft.AspNetCore.DataProtection.Secret.Secret(System.ArraySegment<byte> value) -> void
Microsoft.AspNetCore.DataProtection.Secret.WriteSecretIntoBuffer(System.ArraySegment<byte> buffer) -> void

And probably these, if only for consistency:

static Microsoft.AspNetCore.Cryptography.KeyDerivation.KeyDerivation.Pbkdf2(string! password, byte[]! salt, Microsoft.AspNetCore.Cryptography.KeyDerivation.KeyDerivationPrf prf, int iterationCount, int numBytesRequested) -> byte[]!
Microsoft.AspNetCore.DataProtection.IPersistedDataProtector.DangerousUnprotect(byte[]! protectedData, bool ignoreRevocationErrors, out bool requiresMigration, out bool wasRevoked) -> byte[]!
Microsoft.AspNetCore.DataProtection.Secret.Secret(byte[]! value) -> void
Microsoft.AspNetCore.DataProtection.ITimeLimitedDataProtector.Protect(byte[]! plaintext, System.DateTimeOffset expiration) -> byte[]!
Microsoft.AspNetCore.DataProtection.ITimeLimitedDataProtector.Unprotect(byte[]! protectedData, out System.DateTimeOffset expiration) -> byte[]!
static Microsoft.AspNetCore.DataProtection.DataProtectionAdvancedExtensions.Protect(this Microsoft.AspNetCore.DataProtection.ITimeLimitedDataProtector! protector, byte[]! plaintext, System.TimeSpan lifetime) -> byte[]!

This is a pretty big change, but we'll give it further consideration in 9.0.

Just weighing in to support this proposal. Right now I am writing some serialised data to a cookie which I'd like to protect. While I can do much of the serialisation without allocating by using rented arrays and spans, etc., as soon as I need to protect that data, I'm forced to allocate the byte[]. I'd second @KLuuKer's idea of having this via TryXyz methods so the destination span can also be passed in.

I am working on Antiforgery improvements and it uses IDataProtector under the hood:

We can improve there only if we make a change in DataProtection layer: so far for example validation scenario in Antiforgery cant be impacted due to Protect and Unprotect receiving and returning a byte array.

Proposal

I reviewed all the new APIs we have to expose and here they are:

namespace Microsoft.AspNetCore.DataProtection;

public interface IDataProtector : IDataProtectionProvider
{
+    bool TryProtect(ReadOnlySpan<byte> plainText, Span<byte> destination, out int bytesWritten);
+    bool TryUnprotect(ReadOnlySpan<byte> protectedData, Span<byte> destination, out int bytesWritten);
}

IDataProtector calls IAuthenticatedEncryptor internally, therefore:

namespace Microsoft.AspNetCore.DataProtection.AuthenticatedEncryption;

public interface IAuthenticatedEncryptor
{
+    bool TryDecrypt(ReadOnlySpan<byte> cipherText, ReadOnlySpan<byte> additionalAuthenticatedData, Span<byte> destination, out int bytesWritten);
+    bool TryEncrypt(ReadOnlySpan<byte> plaintText, ReadOnlySpan<byte> additionalAuthenticatedData, Span<byte> destination, out int bytesWritten);
}

Other APIs

I reviewed proposition by @amcasey and I think those can be left for now (ArraySegment<byte> is somewhat similar to Span<byte> and we can get perf improvement without changing those to Span<byte> explicitly. For example we managed to squeeze perf in byte[] overloads without changing ArraySegment<byte>:

• Microsoft.AspNetCore.DataProtection.ISecret.WriteSecretIntoBuffer(System.ArraySegment buffer) -> void
• Microsoft.AspNetCore.DataProtection.Secret.Secret(System.ArraySegment value) -> void
• Microsoft.AspNetCore.DataProtection.Secret.WriteSecretIntoBuffer(System.ArraySegment buffer) -> void

these ones should be also recreated with Span overload for consistency, but personally we can do that in waves / per request probably?

• static Microsoft.AspNetCore.Cryptography.KeyDerivation.KeyDerivation.Pbkdf2(string! password, byte[]! salt, Microsoft.AspNetCore.Cryptography.KeyDerivation.KeyDerivationPrf prf, int iterationCount, int numBytesRequested) -> byte[]!
• static Microsoft.AspNetCore.DataProtection.DataProtectionAdvancedExtensions.Protect(this Microsoft.AspNetCore.DataProtection.ITimeLimitedDataProtector! protector, byte[]! plaintext, System.TimeSpan lifetime) -> byte[]!

there are also other types of IDataProtector, but again - can come later per request, because implementing so many new API methods will result in a huge PR.

• Microsoft.AspNetCore.DataProtection.ITimeLimitedDataProtector.Protect(byte[]! plaintext, System.DateTimeOffset expiration) -> byte[]!
• Microsoft.AspNetCore.DataProtection.ITimeLimitedDataProtector.Unprotect(byte[]! protectedData, out System.DateTimeOffset expiration) -> byte[]!
• Microsoft.AspNetCore.DataProtection.IPersistedDataProtector.DangerousUnprotect(byte[]! protectedData, bool ignoreRevocationErrors, out bool requiresMigration, out bool wasRevoked) -> byte[]!

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

• The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
• The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
• Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

concerns from @BrennanConroy:

1. adding members to interface is a breaking change and especially for netstandard due to default interface implementations, so cover with #if NETCOREAPP for sure
2. default implementation should exist to call Protect from TryProtect due to possible user custom implementation which we would call eventually (to not break user)

• What scenarios are we trying to improve?
  • Antiforgery token generation and validation.
  • Are we introducing DIMs for this?
    • Yes. For .NET 10, we'll allocate and call the existing array-based overloads. We'll #ifdef out for netstandard.
• Is the bool return sufficient? Should we use the operation status pattern to indicate how large the destination buffer needs to be?
  • Do these methods only return false if the destination is too small?
  • Do we need a second out parameter for bytes required? Can we combine these out parameters considering they are mutually exclusive.
    • No. We looked at IPAddress.TryWrite and BigInteger.TryWrite and they don't do this.
    • We also looked at variaous OperationStatus encoder APIs that don't appear to have multiple out parameters for bytes written.
• Can we add a GetProtectedSize(ReadOnlySpan<byte> plainText) and GetUnprotectedSize(ReadOnlySpan<byte> cypherText)?
  • Maybe? This needs investigation.
  • Would this change the design of anything else?

This is looking good, but we want to have a better understanding if we need the GetProtectedSize(ReadOnlySpan<byte> plainText) and GetUnprotectedSize(ReadOnlySpan<byte> cypherText) APIs before giving final approval.

public interface IDataProtector : IDataProtectionProvider
{
+ #if NET
+    bool TryProtect(ReadOnlySpan<byte> plainText, Span<byte> destination, out int bytesWritten);
+    bool TryUnprotect(ReadOnlySpan<byte> protectedData, Span<byte> destination, out int bytesWritten);
+ #endif
}

public interface IAuthenticatedEncryptor
{
+ #if NET
+    bool TryDecrypt(ReadOnlySpan<byte> cipherText, ReadOnlySpan<byte> additionalAuthenticatedData, Span<byte> destination, out int bytesWritten);
+    bool TryEncrypt(ReadOnlySpan<byte> plaintText, ReadOnlySpan<byte> additionalAuthenticatedData, Span<byte> destination, out int bytesWritten);
+ #endif
}