Argthtjtr

To migrate "trees of related data" from an external system to Salesforce I am using dynamic chains of Apex Queueables. Imagine I am "synchronizing" Accounts, Opportunities and Contacts from an external CRM into Salesforce.

There is a separate Queueable class for each object type and to stay within limits each Queueable is just migrating a certain amount of objects. This is how a typical flow could look like:

1. 
   AccountQueueable: Get 10 Accounts


2. 
   ContactQueueable: Get 1000 Contacts of that 10 Accounts



3. OpportunityQueueable: Get 200 Opptys from Contacts and Accounts in 1./2.




4. AccountQueueable: Rerun for next 10 accounts



5. 
   ContactQueueable: Get related Contacts
   ...you get the scheme

This works with 3 related object types but if I get more I am unable to run even the smallest scenario in my dev org because I hit the limit document here https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_queueing_jobs.htm

For Developer Edition and Trial organizations, the maximum stack depth
for chained jobs is 5, which means that you can chain jobs four times
and the maximum number of jobs in the chain is 5, including the
initial parent queueable job.

I can't use Batch as I am not working on a single object. I also can't increase the limit as Salesforce told me that's a hard limit of dev orgs.

Maybe my overall approach is wrong?! What would you do here?

I'd say use a batchable class. What you need is a dynamic approach. Even though you're working with multiple objects, a batch class can still be used here. Here's a design pattern for you:

public class DynamicBatch implements Database.Batchable<batchAction>, Database.Stateful {

  class StateInfo {

    public Account accounts = new Account[0];

    public Contact contacts = new Contact[0];

    public Opportunity opps = new Opportunity[0];

    // ...

  }

  StateInfo state = new StateInfo();

  interface batchAction {

    void execute(StateInfo state) {

  }

  class AccountAction implements BatchAction {

    void execute(StateInfo state) {

      // ...

    }

  }

  class ContactAction implements BatchAction {

    void execute(StateInfo state) {

      // ...

    }

  }

  class OpportunityAction implements BatchAction {

    void execute(StateInfo state) {

      // ...

    }

  }

  public batchAction start(Database.BatchableContext context) {

    return new batchAction { new AccountAction(), new ContactAction(), new OpportunityAction() };

  }

  public void execute(Database.BatchableContext context, batchAction scope) {

    scope[0].execute(state);

  }

  public void finish(Database.BatchableContext context) {

    if(!finished()) {

      Database.executeBatch(new DynamicBatch());

    }

  }

  // ...

}

You can adjust this as you like, but hopefully you get the general idea. This batch class is called with a scope size of 1. This behaves like an unkillable Queueable and can be chained indefinitely, unlike Queueable calls. This also avoids "hacks" like swapping back and forth between future/queueable or some other design.

If you run a Batchable from Account, the execute method could query all the necessary child records and act accordingly. If the transaction limits would get busted there, spawn a Queueable from the batch. You won't reach maximum stack depth unless any one Queueable launched by the batch passes 5 deep.

Well, I have a hack

We cant call future from future, but we can call Future from a Queueable and Queueable from future.

So from the 5th Queuable call the future, and then that Future can call another Queueable to have an infinite chain in Developer orgs.

Edit: I did a demo of recursion, calling Future from Queuable and from Queuable call future, I was able to chain over 400+ levels deep before my Daily Async Apex Limit ended.