Documentation Request: addEntities in core/data and how to register custom endpoints

[acketon]

What problem does this address?

I've been trying to better understand how to register a custom endpoint added in PHP and make it available to use in the block editor. I've developed a good understanding of how to fetch data with useSelect() from existing data sources such as posts or pages. But I need to go beyond that in cases where the existing endpoints don't give us the option of returning data we need or querying in complex ways.

For instance let's say I need to query for something more complex such as page posts with a specific template applied that also have another specific post meta field entered and perhaps are also tagged with a specific term. In PHP I can do this with WP_Query. I've been trying to learn how to do the same in the block editor.

One approach I've been investigating is registering my own custom endpoint and then figuring out how to access that endpoint in the block editor with useSelect() and getEntityRecords.

I could not find much in the way of official documentation about this. Nor much in the community. I was however able to use some AI tools to experiment and put together a working prototype. It seems the key is using addEntities from core/data package to register the custom endpoint.

But I can't find much documentation on addEntities and I'm not sure I would have been able to figure it out relying just on the docs.

What is your proposed solution?

As someone trying to learn this and migrate my knowledge from the PHP side to the React/JS side I'd like to see more documentation in the core data package and specifically with addEntities and how to make custom endpoints available in the editor to use with existing data functions.

I also am unsure how best or when is best to actually call or dispatch a entity config to addEntities. I currently have it in a separate JS file enqueued with the admin_enqueue_scripts hook and use a subscribe approach to subscribe to the coreStore and then dispatch an array of entities like so:
dispatch(coreStore).addEntities(entitiesToRegister);

Maybe also some more documentation on Entities as a concept?

[Mamaduka]

The addEntities was regarded as an advanced API, but it seems more people want to use it for their projects. It might be time to provide a better public API, which doesn't require knowledge of data package internals. cc @WordPress/gutenberg-core

@acketon, the issue title is a bit misleading.t, I initially thought you were also requesting documentation for REST API custom endpoints, which can be found here - https://developer.wordpress.org/rest-api/.

[acketon]

Thanks @Mamaduka. I appreciate that.

I'm somewhat familiar with the REST API. I've used both the core endpoints and custom endpoints primarily to query for posts, meta, etc in the early days of the Gutenberg plugin (WP 4.9).

However at that time on the block side we were using api fetch to grab posts from the custom endpoint.

These days I've been trying to switch a lot over to using the core data package and understand it better. Been using getEntityRecords a lot and getting comfortable with that. We just built a Post Chooser component on top of that to use in our blocks and let site admins easily find and select posts for various use cases.

But we ran into limits when we need to get posts from an endpoint using getEntityRecords with a more complex query than can be done on the core endpoints. For example getting a subset of posts that have specific post meta field values.

That led me to try to figure out if there was support yet to query a custom rest api endpoint we might register in a project via core data functions in the Block Editor like getEntityRecords.

I think I have a prototype working now by using addEntities() in core data to register my custom route but it was difficult figuring out with the state of the documentation on it.

If someone with expertise/experience with the core data package could improve the docs for it I think it would be helpful for my whole team trying to transition from primarily PHP to using similar advanced features in the Block Editor.

Unless I'm off course here, it seems addEntities() is key for this?

[Mamaduka]

Unless I'm off course here, it seems addEntities() is key for this?

You're totally right. Currently, the addEntities action is the only available option for registering new entities.

@ndiego, I think you've been an early adopter in this area. Do you have any tips and tricks to share as a consumer?

[acketon]

I thought it might be helpful to add a bit more background on why this is important to us. This is a novel, sorry.

I'm at Boston University where we have a large WordPress multi-site, multi-network environment. About 125+ custom themes, and about 2,000+ WordPress sites. Many of our sites are very large, thousands of posts, lots of data and content. Because of this we heavily favor a strategy of keeping content in posts and meta. We use both core post types and custom post types in as structured of an architecture as possible.

Then in the plugins and content components (pre-blocks) we build we favor having site admins select posts and we pull meta from the selected post to display it.

So for example we have a plugin that creates a post type for profiles (Faculty, Staff, Student, Alumni bios). We want our site admins to store a bio/profile about a professor once in a profile post and then when they want to link to or feature that professor in another page (like a page about the academic program they teach, or in a news story about some research they did) we have content components that use a post chooser to let them search all of the profiles for that person's name and select it. Then the content component grabs the appropriate meta from that post and displays it however we want. Often the title, excerpt, image, and some custom meta like position title.

What we don't want to happen is have a site admin create their own "featured faculty card" out of core blocks or raw HTML and manually enter the person's name, photo, title, etc. On a large site with thousands of posts these can quickly get out of date & style.

That's just one example post type. Another common one we have is Statistics which need to be kept accurate across a site but are repeated everywhere. We now have a plugin that centralizes that into a post type so the stat is entered once, but then can be selected and displayed throughout other pages and content. Update it once in one bu-stat post type and you can be assured you've updated your ranking stat everywhere in the site.

Now as we are converting slowly to a block based architecture we need to create those equivalent tools in the block editor. We've been building a custom post chooser component in React on top of getEntityRecords(). We'll be able to use it as a component in blocks we build in our plugins and themes to let site admins search for and select a post. This is working well with the core endpoints / entities for the most part.

But recently we've been getting to the point where I think we need to have our post chooser or other components and hooks that are built on top of useSelect() and getEntityRecords(), etc use a custom endpoint as well. We have more complicated scenarios where we need to query for posts in a post type that the Core endpoints don't support without modification. Finding all bu-profile posts that have a post meta field set to some value for instance. Hypothetically a profile might have post meta that lists published papers and we want to be able to return a list of profiles in a block that are only those that have published papers meta filled out.

That led us to trying to see how / if we could get a custom endpoint we create working in the block editor with useSelect() and getEntityRecords(). The goal would be to be able to have the luxury of passing a custom endpoint to our Post Chooser component as a prop when needed for a more unique problem. But otherwise have the Post Chooser use getEntityRecords() on the core endpoints for simpler post queries/search/etc.

<PostChooser
	onSelectPost={ calloutSelectedPostHandler }
	onClose={ () => setIsPostChooserModalOpen( false ) }
	postTypes={ [
		{ label: __( 'Profiles' ), value: 'bu-profile' },
	] }
	primaryPostType="bu-profile"
/>

or with a custom endpoint:

<PostChooser
	onSelectPost={ calloutSelectedPostHandler }
	onClose={ () => setIsPostChooserModalOpen( false ) }
	postTypes={ [
		{ label: __( 'Profiles' ), value: 'bu-profile' },
	] }
	primaryPostType="bu-profile"
	endpoint={ [
		{ 
			name: 'bu-profile', 
			kind: 'bu-profiles/v1',
			query: 'some-meta-query-object-here'
		},
	] }
/>

What I'd love to see documentation on is how .addEntities() is best used, what should be passed to it in an entity configuration. Ideally with some examples of a hypothetical custom endpoint. From what I've learned it seems something like this:

/**
 * Configuration for custom entities
 *
 * This array defines all custom entities to be registered.
 * Add new entities here following the same structure.
 */
const customEntities = [
    {
        // Required properties
        name: 'import-bob-custom',      // Entity name - used in selectors
        kind: 'bu-custom/v1',           // Entity kind - namespace
        baseURL: '/bu-custom/v1/import-bob', // REST API endpoint path
        route: '/bu-custom/v1/import-bob/(?P<id>[\\d]+)', // Single item route pattern (for getEntityRecord)

        // Optional properties
        plural: 'import-bob-customs',   // Plural form for dynamic selector naming
        label: 'Import Bob Custom',     // Human-readable label

        // Define the type of identifier used (id, slug, or custom)
        idType: 'id',                  // Identifies the parameter type in the single item endpoint
    },
    // Example of how to add additional entities:
    
    {
        name: 'another-custom-entity',
        kind: 'bu-custom/v1',
        baseURL: '/bu-custom/v1/another-endpoint',
        plural: 'another-custom-entities',
        label: 'Another Custom Entity',
    }
    
];

And then also documentation on when and how best to actually call addEntities(). I'm unclear on when in the sequence of the block editor loading in the admin would be best to add our custom Entities. It would have to be after some of the core data package has loaded but before any of the blocks are loaded and try querying for data. I've tested it in a JS file enqueued on the hook admin_enqueue_scripts but I'm not sure if that's best.

And then in that JS file that's enqueued I've tried using subscribe() from @wordpress/data to check if the coreStore is loaded and available and then if so call dispatch(coreStore).addEntities(entitiesToRegister); to register the entities.

I'm unclear what the preferred method and timing of registering the custom entities would be so it would be nice if the docs could touch on some of that.