This is what I get out now, as Markdown, when testing the minimal branch. I don't know if that helps, but you can see what I'm sort of concentrating on.
The tests for other relays would just be a subset of these, for the websocket and latency, I guess. I figured I'd start with realy and then reduce the test set accordingly.
Tell me, if I'm testing this write and if I'm missing tests. Some are probably already passing on a different branch, but I wanted to test something that fails a lot, to see how the report ends up looking. Pass is an easier print.
# Realy Backend Evaluation Report
> **Generated:** 2025-06-28T07:51:10.568Z
> **Target:** `http://localhost:3334`
> **Backend Type:** Realy
> **Authentication:** ✅ Available
> **Test Environment:** Standard
## Executive Summary
This report provides a comprehensive evaluation of the **Realy** Nostr relay backend. The testing was conducted using automated test suites designed to validate core functionality, standards compliance, and performance characteristics.
### Overall Results
| Metric | Value |
|--------|-------|
| **Total Tests** | 14 |
| **🟢 Passed** | 9 |
| **🟡 Partial** | 2 |
| **🔴 Failed** | 3 |
| **Success Rate** | **71.4%** |
### Test Category Breakdown
| Category | Tests | 🟢 Pass | 🟡 Partial | 🔴 Fail | Success Rate |
|----------|-------|---------|------------|---------|--------------|
| **Connectivity** | 3 | 1 | 2 | 0 | ⚠️ **66.7%** |
| **Core** | 3 | 3 | 0 | 0 | ✅ **100.0%** |
| **Standards** | 1 | 1 | 0 | 0 | ✅ **100.0%** |
| **Performance** | 1 | 0 | 0 | 1 | ❌ **0.0%** |
## Detailed Test Results
### Connectivity Tests
#### 🟢 basicConnectivity
**Status:** PASS
**Description:** Validates that the relay is accessible and responding to basic HTTP requests
**Details:** Relay responded with 426 Upgrade Required (expected for HTTP requests)
**Technical Data:**
```json
{
"statusCode": 426
}
```
#### 🟡 webSocket
**Status:** PARTIAL
**Description:** Tests WebSocket connection and message handling capabilities
**Details:** Connected successfully, sent 2 messages, received 0 responses
**Technical Data:**
```json
{
"connected": true,
"messagesSent": 2,
"messagesReceived": 0,
"errors": []
}
```
**Recommendations:**
- Ensure WebSocket support is enabled in the relay
- Check WebSocket upgrade handling
- Verify message format compliance with Nostr protocol
- Review WebSocket connection logs
#### 🟡 serverSentEvents
**Status:** PARTIAL
**Description:** Validates Server-Sent Events (SSE) streaming functionality
**Details:** Connected successfully, received 0 events
**Test Steps:**
#### Authentication
**Status:** PASS
**Details:** NIP-98 token generated successfully
**Data:** ```json
{
"hasAuth": true
}
```
#### SSE Connection
**Status:** PASS
**Details:** Successfully connected to SSE endpoint
**Data:** ```json
{
"statusCode": "422"
}
```
#### Event Reception
**Status:** PARTIAL
**Details:** Received 0 events
**Data:** ```json
{
"eventsReceived": 0
}
```
**Technical Data:**
```json
{
"connected": true,
"eventsReceived": 0,
"statusCode": "422",
"errors": []
}
```
**Recommendations:**
- Enable SSE support in the relay configuration
- Check CORS settings for SSE endpoints
- Verify proper event stream formatting
- Test SSE endpoint accessibility
---
### Core Tests
#### 🟢 eventCount
**Status:** PASS
**Description:** Checks the relay's ability to report total event count
**Details:** Database contains 13 events
**Technical Data:**
```json
{
"statusCode": 200,
"eventCount": 13
}
```
#### 🟢 eventPublishing
**Status:** PASS
**Description:** Validates event publishing and storage functionality
**Details:** Event published successfully
**Technical Data:**
```json
{
"eventId": "168cbd2ad3e73ca7736752e048b2f97b11fa6673c3a02d56360f82b32be7a61d",
"pubkey": "fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1",
"response": "\"event accepted\""
}
```
#### 🟢 apiDocumentation
**Status:** PASS
**Description:** Checks availability of API documentation endpoints
**Details:** API documentation endpoint is accessible
**Technical Data:**
```json
{
"statusCode": 200,
"contentLength": 392
}
```
---
### Standards Tests
#### 🟢 nip11
**Status:** PASS
**Description:** Tests NIP-11 relay information endpoint compliance
**Details:** NIP-11 relay information endpoint working
**Technical Data:**
```json
{
"statusCode": 200,
"relayInfo": {
"name": "realy-1",
"description": "relay powered by the realy framework",
"supported_nips": [
1,
4,
9,
11,
12,
15,
16,
20,
33,
40,
42,
65,
70
],
"software": "https://realy.lol",
"version": "v1.18.0",
"limitation": {
"max_limit": 512,
"auth_required": false,
"payment_required": false,
"restricted_writes": false
},
"icon": "https://cdn.satellite.earth/ac9778868fbf23b63c47c769a74e163377e6ea94d3f0f31711931663d035c4f6.png"
}
}
```
---
### Performance Tests
#### 🔴 concurrentStreaming
**Status:** FAIL
**Description:** Tests concurrent event streaming and publishing
**Details:** 1/3 subtests passed
**Test Steps:**
#### SSE Streaming Test
**Status:** FAIL
**Details:** SSE events: 0, published events: 3
**Data:** ```json
{
"sseEvents": 0,
"publishedEvents": 3,
"errors": []
}
```
#### WebSocket-to-REST Test
**Status:** PASS
**Details:** Event published via WebSocket and retrieved via REST API
**Data:** ```json
{
"eventId": "d6b96f4cae6d61a296113f2f7c1a4e1bc3ee1c2864c4fb0394da28124ee0d5fe",
"wsPublished": true,
"restRetrieved": true,
"retrievedEvent": {
"id": "d6b96f4cae6d61a296113f2f7c1a4e1bc3ee1c2864c4fb0394da28124ee0d5fe",
"pubkey": "fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1",
"created_at": 1751097039,
"kind": 1,
"tags": [
[
"t",
"ws-rest-test"
],
[
"p",
"fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1"
]
],
"content": "WebSocket-to-REST test event - 2025-06-28T07:50:39.225Z",
"sig": "4a110be6162f5db93bfadc30f9686b6267b968f6af7a6e254c1c11943fb080020928b2886b9581f4e953faefbd98d6943d6fd966844f35c49ba2a645776d8c2c"
}
}
```
#### REST-to-WebSocket Test
**Status:** FAIL
**Details:** Event published via REST API but not received via WebSocket: WebSocket subscription timeout
**Data:** ```json
{
"eventId": "6c7c59bf3fb2020c1018ccbbe8807773eb8963cc492b5ff057b6182a0346d0fb",
"restPublished": true,
"restResult": "\"event accepted\"",
"wsRetrieved": false,
"reason": "WebSocket subscription timeout"
}
```
**Technical Data:**
```json
{
"stepsPassed": 1,
"totalSteps": 3,
"subtests": {
"sseStreaming": false,
"wsToRest": true,
"restToWs": false
}
}
```
**Recommendations:**
- Review the specific error details provided above
- Check relay logs for related error messages
- Verify configuration settings
- Test with different parameters or conditions
---
### Other Tests
#### 🟢 eventIdsAndRetrieval
**Status:** PASS
**Details:** Completed 3/3 steps successfully.
**Technical Data:**
```json
{
"stepsPassed": 3,
"totalSteps": 3,
"eventsRetrieved": 5
}
```
#### 🔴 httpLatency
**Status:** FAIL
**Details:** Average latency: 3.00ms, 2/3 endpoints successful
**Technical Data:**
```json
{
"averageLatency": 3,
"successfulTests": 2,
"totalTests": 3,
"results": [
{
"endpoint": "Base URL",
"latencyMs": 7,
"statusCode": 426,
"success": false
},
{
"endpoint": "API Documentation",
"latencyMs": 3,
"statusCode": 200,
"success": true
},
{
"endpoint": "Event Count",
"latencyMs": 3,
"statusCode": 200,
"success": true
}
]
}
```
#### 🔴 webSocketLatency
**Status:** FAIL
**Details:** Average WebSocket latency: 0.00ms, 0/2 messages successful
**Technical Data:**
```json
{
"averageLatency": 0,
"successfulTests": 0,
"totalTests": 2,
"results": [
{
"message": "latency-test-1",
"latencyMs": null,
"success": false,
"error": "Timeout"
},
{
"message": "latency-test-2",
"latencyMs": null,
"success": false,
"error": "Timeout"
}
]
}
```
#### 🟢 sseLatency
**Status:** PASS
**Details:** First event latency: 6ms
**Technical Data:**
```json
{
"latencyMs": 6,
"received": true,
"threshold": 5000
}
```
#### 🟢 loadPerformance
**Status:** PASS
**Details:** 10/10 concurrent requests successful (100.0%), average latency: 14.40ms
**Technical Data:**
```json
{
"concurrentRequests": 10,
"successfulRequests": 10,
"failedRequests": 0,
"successRate": 100,
"averageLatency": 14.4,
"minLatency": 14,
"maxLatency": 15,
"results": [
{
"requestId": 1,
"latencyMs": 15,
"statusCode": 200,
"success": true
},
{
"requestId": 2,
"latencyMs": 15,
"statusCode": 200,
"success": true
},
{
"requestId": 3,
"latencyMs": 15,
"statusCode": 200,
"success": true
},
{
"requestId": 4,
"latencyMs": 15,
"statusCode": 200,
"success": true
},
{
"requestId": 5,
"latencyMs": 14,
"statusCode": 200,
"success": true
},
{
"requestId": 6,
"latencyMs": 14,
"statusCode": 200,
"success": true
},
{
"requestId": 7,
"latencyMs": 14,
"statusCode": 200,
"success": true
},
{
"requestId": 8,
"latencyMs": 14,
"statusCode": 200,
"success": true
},
{
"requestId": 9,
"latencyMs": 14,
"statusCode": 200,
"success": true
},
{
"requestId": 10,
"latencyMs": 14,
"statusCode": 200,
"success": true
}
]
}
```
#### 🟢 eventPublishingPerformance
**Status:** PASS
**Details:** 5/5 events published successfully (100.0%), average latency: 35.20ms
**Technical Data:**
```json
{
"publishCount": 5,
"successfulPublishes": 5,
"failedPublishes": 0,
"successRate": 100,
"averageLatency": 35.2,
"results": [
{
"eventId": "6184df29c60849ce",
"latencyMs": 30,
"success": true,
"response": "\"event accepted\""
},
{
"eventId": "2ea2510c7f568d79",
"latencyMs": 57,
"success": true,
"response": "\"event accepted\""
},
{
"eventId": "73f7844f948d7796",
"latencyMs": 22,
"success": true,
"response": "\"event accepted\""
},
{
"eventId": "2394f85595708053",
"latencyMs": 25,
"success": true,
"response": "\"event accepted\""
},
{
"eventId": "816d6a67a00ec1d9",
"latencyMs": 42,
"success": true,
"response": "\"event accepted\""
}
]
}
```
---
## Recommendations
1. **High Priority:** Fix event retrieval and filtering - essential for client functionality.
2. **Low Priority:** Consider implementing NIP-98 authentication for enhanced security.
## Technical Details
### Test Configuration
```json
{
"host": "localhost",
"port": 3334,
"hasAuth": true,
"timestamp": "2025-06-28T07:49:58.918Z"
}
```
### Test Environment
- **Test Framework:** GitRepublic Web Relay Testing Suite
- **Test Date:** 2025-06-28T07:51:10.568Z
- **Protocol:** Nostr
- **Backend:** Realy
- **Target URL:** http://localhost:3334
## Conclusion
⚠️ **Moderate Issues** - The Realy backend has several issues that need attention. While some core functionality works, there are significant problems to address.
This report should be used as a guide for improving the relay implementation. For questions about this evaluation, please refer to the GitRepublic Web documentation.
---
*Report generated by GitRepublic Web Relay Testing Suite*
