Solana transactions are limited to 1232 bytes which was too restrictive when I was implementing Solana IBC bridge while working at Composable Foundation. The smart contract had to be able to ingest signed Tendermint block headers which were a few kilobytes in size.

To overcome this obstacle, I’ve used what I came to call chunking. By sending the instruction data in multiple transactions (similarly to the way Solana programs are deployed), the Solana IBC smart contract is capable of working on arbitrarily-large instructions. This article describes how this process works and how to incorporate it with other smart contracts (including those using the Anchor framework).

This article assumes familiarity with Solana; it’s not a step-by-step guide on creating Solana programs. Nevertheless, code samples are from examples available in the solana-write-account repository (a chsum-program Solana program and chsum-client command line tool used to invoke said program) which can be used as a starting point when incorporating the chunking method described below.

Demonstration of the problem

First let’s reproduce the described issue. Consider the following (somewhat contrived) chsum Solana program which computes a simple parameterised checksum. When called, the first byte of its instruction data is the checksum parameter and the rest is the data to calculate the checksum of.

solana_program::entrypoint!(process_instruction);

fn process_instruction<'a>(
    _program_id: &'a Pubkey,
    _accounts: &'a [AccountInfo],
    instruction: &'a [u8],
) -> Result<(), ProgramError> {
    let (mult, data) = instruction
        .split_first()
        .ok_or(ProgramError::InvalidInstructionData)?;
    let sum = data.chunks(2).map(|pair| {
        u64::from(pair[0]) * u64::from(*mult) +
            pair.get(1).copied().map_or(0, u64::from)
    }).fold(0, u64::wrapping_add);
    solana_program::msg!("{}", sum);
    Ok(())
}

The program works on data of arbitrary length which can be easily observed by executing it with progressively longer buffers. However, due to aforementioned transaction size limit, eventually the operation fails:

$ chsum-client 2 ab
Program log: 292

$ chsum-client 2 abcdefghijklmnopqrstuvwxyz
Program log: 4264

$ data=…
$ echo "${#data}"
1062
$ chsum-client 2 "$data"
RPC response error -32602: decoded solana_sdk::transaction::versioned::VersionedTransaction too large: 1233 bytes (max: 1232 bytes)

The write-account program

To solve the problem, the overlarge instruction can be split into smaller chunks which can be sent to the blockchain in separate transactions and stored inside of a Solana account. For this to work two things are needed: i) a smart contract which can receive and concatenate all those chunks; and ii) support in the target smart contract for reading instruction data from an account (rather than from transaction’s payload).

The first requirement is addressed by the write-account program. It copies bytes from its instruction data into a given account at specified offset. Subsequent calls allow arbitrary (and most importantly arbitrarily-long) data to be written into the account.

RPC Client

The simplest way to send the chunks to the smart contract is to use client library functions packaged alongside the write-account program. First, add a new dependency to the RPC client:

[dependencies.solana-write-account]
git = "https://github.com/mina86/solana-write-account"
features = ["client"]

And with that, the WriteIter can be used to split overlong instruction data into chunks and create all the necessary instructions. By default the data is length-prefixed when it’s written to the account. This simplifies reuse of the account since the length of written data can be decoded without a need to resize the account.

// Write chunks to a new account
let (chunks, write_account, write_account_bump) =
    solana_write_account::instruction::WriteIter::new(
        &WRITE_ACCOUNT_PROGRAM_ID,
        signer.pubkey(),
        WRITE_ACCOUNT_SEED,
        data,
    )?;
for inst in chunks {
    send_and_confirm_instruction(client, signer, inst)?;
}

// Invoke the target smart contract. write_account is the
// account with the instruction data.  It will need to be
//  passed to the smart contract as last account.
todo!("Invoke the target smart contract");

// Optionally, free the account to recover deposit
let inst = solana_write_account::instruction::free(
    WRITE_ACCOUNT_PROGRAM_ID,
    signer.pubkey(),
    Some(write_account),
    WRITE_ACCOUNT_SEED,
    write_account_bump,
)?;
send_and_confirm_instruction(client, signer, inst)?;

The data is copied into a Program Derived Address (PDA) account owned by the write-account program. The smart contract makes sure that different signers get their own accounts so that they won’t override each other’s work. WRITE_ACCOUNT_SEED allows a single signer to maintain multiple accounts if necessary.

The address of the account holding the instruction data is saved in the write_account variable. But before it can be passed to the target smart contract, the smart contract needs to be altered to support such calling convention.

Note on parallel execution

With some care, the instructions returned by WriteIter can be executed in parallel thus reducing amount of time spent calling the target smart contract. One complication is that the account may need to be resized when chunks are written into it. Since account can be increased by only 10 KiB in a single instruction, this becomes an issue if trying to write a chunk which is over 10 KiB past the end of the account.

One way to avoid this problem, is to group the instructions and executed them ten at a time. Once first batch executes, the next can be send to the blockchain. Furthermore, if the account is being reused, it may already be sufficiently large. And of course, this is not an issue with the data doesn’t exceed 10 KiB.

The target smart contract

The Solana runtime has no built-in mechanism for passing instruction data from an account. Smart contract needs to explicitly support such calling method. One approach is to always read data from an account. This may be appropriate if the smart contract usually deals with overlong payloads. A more flexible approach is to read instruction from the account if instruction data in the transaction is empty. This can be done by defining a custom entry point:

/// Solana smart contract entry point.
///
/// If the instruction data is empty, reads length-prefixed data
/// from the last account and treats it as the instruction data.
///
/// # Safety
///
/// Must be called with pointer to properly serialised
/// instruction such as done by the Solana runtime.  See
/// [`solana_program::entrypoint::deserialize`].
#[no_mangle]
pub unsafe extern "C" fn entrypoint(input: *mut u8) -> u64 {
    // SAFETY: Guaranteed by the caller.
    let (prog_id, mut accounts, mut instruction_data) = unsafe {
        solana_program::entrypoint::deserialize(input)
    };

    // If instruction data is empty, the actual instruction data
    // comes from the last account passed in the call.
    if instruction_data.is_empty() {
        match get_ix_data(&mut accounts) {
            Ok(data) => instruction_data = data,
            Err(err) => return err.into(),
        }
    }

    // Process the instruction.
    process_instruction(
        prog_id,
        &accounts,
        instruction_data,
    ).map_or_else(
        |error| error.into(),
        |()| solana_program::entrypoint::SUCCESS
    )
}

/// Interprets data in the last account as instruction data.
fn get_ix_data<'a>(
    accounts: &mut Vec<AccountInfo<'a>>,
) -> Result<&'a [u8], ProgramError> {
    let account = accounts.pop()
        .ok_or(ProgramError::NotEnoughAccountKeys)?;
    let data = alloc::rc::Rc::try_unwrap(account.data)
        .ok().unwrap().into_inner();
    if data.len() < 4 {
        return Err(ProgramError::InvalidInstructionData);
    }
    let (len, data) = data.split_at(4);
        .ok_or(ProgramError::InvalidInstructionData)?;
    let len = u32::from_le_bytes(len.try_into().unwrap());
    data.get(..(len as usize))
        .ok_or(ProgramError::InvalidInstructionData)
}

solana_program::custom_heap_default!();
solana_program::custom_panic_default!();

The solana-write-account crate packages all of that code. Rather than copying the above, a smart contract wanting to accept instruction data from an account can add the necessary dependency (this time with the lib Cargo feature enabled):

[dependencies.solana-write-account]
git = "https://github.com/mina86/solana-write-account"
features = ["lib"]

and use entrypoint macro defined there (in place of solana_program::entrypoint macro):

solana_write_account::entrypoint!(process_instruction);

Anchor framework

This gets slightly more complicated for anyone using the Anchor framework. The framework provides abstractions which are hard to break through when necessary. Any Anchor program has to use the anchor_lang::program macro which, among other things, defines the entrypoint function. This leads to conflicts when a smart contract wants to define its own entry point.

Unfortunately, there’s no reliable way to tell Anchor not to introduce that function. To add write-account support to the Solana IBC bridge I had to fork Anchor and extend it with the following change which introduces support for a new custom-entrypoint Cargo feature:

diff --git a/lang/syn/src/codegen/program/entry.rs b/lang/syn/src/codegen/program/entry.rs
index 4b04da23..093b1813 100644
--- a/lang/syn/src/codegen/program/entry.rs
+++ b/lang/syn/src/codegen/program/entry.rs
@@ -9,7 +9,7 @@ pub fn generate(program: &Program) -> proc_macro2::TokenStream {
         Err(anchor_lang::error::ErrorCode::InstructionMissing.into())
     });
     quote! {
-        #[cfg(not(feature = "no-entrypoint"))]
+        #[cfg(not(any(feature = "no-entrypoint", feature = "custom-entrypoint")))]
         anchor_lang::solana_program::entrypoint!(entry);
         /// The Anchor codegen exposes a programming model where a user defines
         /// a set of methods inside of a `#[program]` module in a way similar

The feature needs to be enabled in the Cargo.toml of the Anchor program which wants to take advantage of it:

[features]
default = ["custom-entrypoint"]
custom-entrypoint = []

Conclusion

To achieve a sub-second finality Solana had to introduce significant constraints on the protocol and smart contract runtime environment. However, with enough ingenuity at least some of those can be worked around.

This article introduces a way to overcome the 1232-byte transaction size limit through the use of a write-account helper program and library functions available in solana-write-account repository. The solution is general enough that any Solana program, including those using Anchor framework, can use it.

PS. Interestingly, the Solana size limit also affects how many signatures can be verified in a single transaction. I discuss that problem and its solution in another article.