npm Community Forum (Archive)

The npm community forum has been discontinued.

To discuss usage of npm, visit the GitHub Support Community.

replicate.npmjs.com stream publishing really old events

What I Wanted to Do

Listening to the http://replicate.npmjs.com/registry CouchDB stream to process new package versions being published.

What Happened Instead

It appears over the past couple of days tens of thousands of events have been added to the stream and many of them are out of order (based on modified dates) from extremely old packages. For example, here’s a recent event:

{ seq: 7348806,
  id: 'babel-plugin-wpm',
  changes: [ { rev: '2-f10f60a30cbbc209d606d19c853707a4' } ],
  doc: 
   { _id: 'babel-plugin-wpm',
     _rev: '2-f10f60a30cbbc209d606d19c853707a4',
     name: 'babel-plugin-wpm',
     description: 'Some specific transformers for Web Package Manager',
     'dist-tags': { latest: '0.0.2' },
     versions: { '0.0.1': [Object], '0.0.2': [Object] },
     readme: '# babel-plugin-wpm\n\nSome specific transformer(s) for [Web Package Manager](https://github.com/tnhu/wpm)\n\n## Input\n\n```js\nClass({\n  IndexRoute() {\n  },\n\n  signin() {\n  }\n});\n```\n\n## Output\n\n```js\nClass({\n  constructor: function IndexRoute() {\n  },\n\n  signin: function signin() {\n  }\n});\n```\n\n## Rules\n\n* If there is only one CamelCase method. The method becomes constructor.\n* If there are more than one CamelCase methods, the first CamelCase method becomes constructor.\n\n## Installation\n\n```sh\n$ npm install babel-plugin-wpm\n```\n\n## Usage\n\n### Via `.babelrc` (Recommended)\n\n**.babelrc**\n\n```json\n{\n  "plugins": ["babel-plugin-wpm"]\n}\n```\n\n### Via CLI\n\n```sh\n$ babel --plugins babel-plugin-wpm script.js\n```\n\n### Via Node API\n\n```javascript\nrequire("babel-core").transform("code", {\n  plugins: ["babel-plugin-wpm"]\n});\n```\n',
     maintainers: [ [Object] ],
     time: 
      { modified: '2015-07-04T15:04:24.043Z',
        created: '2015-07-04T07:27:56.302Z',
        '0.0.1': '2015-07-04T07:27:56.302Z',
        '0.0.2': '2015-07-04T15:04:24.043Z' },
     homepage: 'https://github.com/tnhu/babel-plugin-wpm',
     keywords: [ 'babel', 'babel-plugin', 'wpm' ],
     repository: 
      { type: 'git',
        url: 'git+https://github.com/tnhu/babel-plugin-wpm.git' },
     author: { name: 'Tan Nhu' },
     bugs: { url: 'https://github.com/tnhu/babel-plugin-wpm/issues' },
     license: 'MIT',
     readmeFilename: 'README.md' } }

These packages are extremely old and haven’t had any changes in years, yet are showing up under new sequence IDs despite the docs appearing to contain no changes from the previous (year’s old) versions. Are there any issues with the replicate stream at the moment that would explain this behavior? Did npm take any bulk operations that would have dumped all of these old packages back into the replicate stream as new events?

Reproduction Steps

Simply listen to the CouchDB replicate stream.

Details

N/A

Platform Info

N/A


record scratch — Yep! That’s us. We’re currently backfilling https urls in versions.<versions>.dist.url for packages that previously pointed at http://. We’ve ratelimited this job to reduce load on our infrastructure, so these events will come in for a few more days. Sorry for the trouble!


No worries - just wanted to make sure I wasn’t going crazy. Thanks for confirming!