Skip to content

#1141 Set Up SearchClient 🔍 #1021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
demariadaniel wants to merge 20 commits into
base: main
Choose a base branch
from
+1,435 −1,145
Open

#1141 Set Up SearchClient 🔍 #1021

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
63a21aa
Basic SearchClient, w/ some type & import cleanup
demariadaniel Jan 29, 2026
dbcce45
Working Test Client setup
demariadaniel Jan 30, 2026
6529046
Additional Type cleanup
demariadaniel Jan 30, 2026
ca8179a
Basic check for version.distribution, remove console.log
demariadaniel Jan 30, 2026
1acb1ef
Updated Client Init, with corrected use of Promises & Types
demariadaniel Feb 10, 2026
a774721
Refine Types and Function definitions
demariadaniel Feb 11, 2026
92b3f74
Separate Types file & update imports
demariadaniel Feb 11, 2026
f3ca922
Types Setup
demariadaniel Feb 23, 2026
a89be80
Standardized ArrangerSearchClient draft
demariadaniel Feb 23, 2026
0c25998
Cleanup: add elastic options, move type definitions
demariadaniel Feb 23, 2026
eaf5f6e
Major Config Update
demariadaniel Feb 24, 2026
5903fd3
Updated Config definitions
demariadaniel Mar 2, 2026
19223d3
Fixed Client Version bug
demariadaniel Mar 3, 2026
9ad4669
Fixed Testing Round 1, Revised EsSearch definition
demariadaniel Mar 9, 2026
745525b
Fix aggregation integration test issues
demariadaniel Mar 9, 2026
8da16c0
Prettier + Test Config change
demariadaniel Mar 9, 2026
59d1828
Export searchClient
demariadaniel Mar 16, 2026
1bc23e6
Additional setup for methods used in HCMI
demariadaniel Mar 16, 2026
4cb254e
Corrected method definitions
demariadaniel Mar 19, 2026
dcea216
Fix deleteByQuery
demariadaniel Mar 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,6 @@ lerna-debug.log
!**/.env.schema
**/package-lock.json

*.tsbuildinfo
*.tsbuildinfo
# TODO: Remove after testing
/modules/server/testConfigs*
103 changes: 53 additions & 50 deletions integration-tests/admin/test/index.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
import { after, before, describe } from 'node:test';

Copy link
Contributor Author

@demariadaniel demariadaniel Feb 5, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up that multiple files include simple prettier cleanup changes to clear up the repo (mostly order of imports, whitespace, & falsy handling) or Typescript updates to declutter the repo.

The main change is the creation of a SearchClient which replaces instances of ElasticSearch Client.

import { Client } from '@elastic/elasticsearch';
import express from 'express';

import Arranger, { adminGraphql } from '../../../modules/server/dist';
import ajax from '../../../modules/server/dist/utils/ajax';
import addProject from './addProject';
import Arranger, { adminGraphql } from '../../../modules/server/dist/index.js';
import ajax from '../../../modules/server/dist/utils/ajax.js';

import addProject from './addProject.js';

const file_centric_mapppings = require('./assets/file_centric.mappings.json');

Expand All @@ -19,58 +22,58 @@ const app = express();

const api = ajax(`http://localhost:${port}`);
const esClient = new Client({
...(useAuth && {
auth: {
username: esUser,
password: esPwd,
},
}),
node: esHost,
...(useAuth && {
auth: {
username: esUser,
password: esPwd,
},
}),
node: esHost,
});

const cleanup = () =>
Promise.all([
esClient.indices.delete({
index: esIndex,
}),
esClient.indices.delete({
index: 'arranger-projects*',
}),
]);
Promise.all([
esClient.indices.delete({
index: esIndex,
}),
esClient.indices.delete({
index: 'arranger-projects*',
}),
]);

describe('@arranger/admin', () => {
let server;
const adminPath = '/admin/graphql';
before(async () => {
console.log('===== Initializing Elasticsearch data =====');
try {
await cleanup();
} catch (err) {}
await esClient.indices.create({
index: esIndex,
body: file_centric_mapppings,
});
let server;
const adminPath = '/admin/graphql';
before(async () => {
console.log('===== Initializing Elasticsearch data =====');
try {
await cleanup();
} catch (err) {}
await esClient.indices.create({
index: esIndex,
body: file_centric_mapppings,
});

console.log('===== Starting arranger app for test =====');
const router = await Arranger({ esHost, enableAdmin: false });
const adminApp = await adminGraphql({ esHost });
adminApp.applyMiddleware({ app, path: adminPath });
app.use(router);
await new Promise((resolve) => {
server = app.listen(port, () => {
resolve();
});
});
});
after(async () => {
server?.close();
await cleanup();
});
console.log('===== Starting arranger app for test =====');
const router = await Arranger({ esHost, enableAdmin: false });
const adminApp = await adminGraphql({ esHost });
adminApp.applyMiddleware({ app, path: adminPath });
app.use(router);
await new Promise((resolve) => {
server = app.listen(port, () => {
resolve();
});
});
});
after(async () => {
server?.close();
await cleanup();
});

const env = {
api,
esIndex,
adminPath,
};
addProject(env);
const env = {
api,
esIndex,
adminPath,
};
addProject(env);
});
157 changes: 58 additions & 99 deletions integration-tests/server/test/index.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
import { after, before, suite } from 'node:test';

import { Client } from '@elastic/elasticsearch';
import Arranger from '@overture-stack/arranger-server';
import ajax from '@overture-stack/arranger-server/dist/utils/ajax.js';
import express from 'express';

// import { print } from 'graphql';
// import gql from 'graphql-tag';
// import Arranger, { adminGraphql } from '../../../modules/server/dist';
import getSearchClient from '../../../modules/server/src/searchClient/index.js';

// test modules
import data from './assets/model_centric.data.json';
Expand All @@ -26,6 +23,7 @@ const setsIndex = process.env.ES_ARRANGER_SET_INDEX || 'arranger-sets-testing';
const esPwd = process.env.ES_PASS;
const esUser = process.env.ES_USER;
const port = process.env.PORT || 5678;
const clientType = process.env.SEARCH_CLIENT;

const useAuth = !!esPwd && !!esUser;

Expand All @@ -36,14 +34,15 @@ const api = ajax(`http://localhost:${port}`, {
endpoint: '/graphql',
});

const esClient = new Client({
const esClient = await getSearchClient({
...(useAuth && {
auth: {
username: esUser,
password: esPwd,
},
}),
node: esHost,
clientType,
});

const cleanup = () => {
Expand All @@ -61,110 +60,70 @@ suite('integration-tests/server', () => {
let server;
const documentType = 'model';

before(async () => {
console.log('\n(Initializing Elasticsearch and Arranger)');
before(
async () => {
console.log('\n(Initializing Elasticsearch and Arranger)');

try {
await cleanup();
} catch (err) {
//
}
try {
await cleanup();
} catch (err) {
//
}

await esClient.indices.create({
index: esIndex,
body: mappings,
});

for (const datum of data) {
await esClient.index({
await esClient.indices.create({
index: esIndex,
id: datum._id,
body: datum._source,
refresh: 'wait_for',
body: mappings,
});
}

try {
const router = await Arranger({
// needed to see the mapping
enableAdmin,
// This may be useful when troubleshooting tests
enableLogs: true,
esHost,
getServerSideFilter: () => ({
op: 'not',
content: [
{
op: 'in',
content: {
fieldName: 'access_denied',
value: ['true'],
for (const datum of data) {
await esClient.index({
index: esIndex,
id: datum._id,
body: datum._source,
refresh: 'wait_for',
});
}

try {
const router = await Arranger({
// needed to see the mapping
enableAdmin,
// This may be useful when troubleshooting tests
enableLogs: true,
esHost,
getServerSideFilter: () => ({
op: 'not',
content: [
{
op: 'in',
content: {
fieldName: 'access_denied',
value: ['true'],
},
},
},
],
}),
setsIndex,
});

app.use(router);
],
}),
setsIndex,
});

// TODO: reenable once Admin is back online
// const adminApp = await adminGraphql({ esHost });
// adminApp.applyMiddleware({ app, path: adminPath });
app.use(router);

await new Promise((resolve) => {
server = app.listen(port, () => {
resolve(null);
await new Promise((resolve) => {
server = app.listen(port, () => {
resolve(null);
});
});
});

// TODO: reenable once Admin is back online
// /**
// * uses the admin API to adds some metadata
// */
// await api.post({
// endpoint: adminPath,
// body: {
// query: print(gql`
// mutation($projectId: String!) {
// newProject(id: $projectId) {
// id
// __typename
// }
// }
// `),
// variables: {
// projectId,
// },
// },
// });

// await api.post({
// endpoint: adminPath,
// body: {
// query: print(gql`
// mutation($projectId: String!, $documentType: String!, $esIndex: String!) {
// newIndex(projectId: $projectId, documentType: $documentType, esIndex: $esIndex) {
// id
// }
// }
// `),
// variables: {
// projectId,
// documentType,
// esIndex,
// },
// },
// });

console.log('******* Starting tests *******');
} catch (err) {
console.error('error:', err);
throw err;
}
}, {
timeout: 10000,
});
console.log('******* Starting tests *******');
} catch (err) {
console.error('error:', err);
throw err;
}
},
{
timeout: 10000,
},
);

after(async () => {
try {
Expand Down
9 changes: 3 additions & 6 deletions integration-tests/server/test/manageSets.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ export default ({ api, documentType }) => {

assert.equal(data.errors, undefined);

setId = data.data.newSet.setId;
setId = data?.data?.newSet?.setId;
});

test('2.retrieves newly created set successfully', async () => {
Expand Down Expand Up @@ -55,11 +55,8 @@ export default ({ api, documentType }) => {
});

assert.equal(data.errors, undefined);
const allSetIds = data.data.sets.hits.edges.map(({ node }) => node.setId);
const allSetIds = data?.data?.sets?.hits?.edges.map(({ node }) => node.setId) || [];

assert.ok(
allSetIds.includes(setId),
`Expected [${allSetIds.join(', ')}] to include ${setId}`
);
assert.ok(allSetIds.includes(setId), `Expected [${allSetIds.join(', ')}] to include ${setId}`);
});
};
Loading