# sppull

[![NPM](https://camo.githubusercontent.com/0163d453c74b8d0249a086f36bce6a006e145a70/68747470733a2f2f6e6f6465692e636f2f6e706d2f737070756c6c2e706e673f6d696e693d7472756526646f776e6c6f6164733d7472756526646f776e6c6f616452616e6b3d747275652673746172733d74727565)](https://nodei.co/npm/sppull/)

[![npm version](https://camo.githubusercontent.com/2cc81d9ac22c5e23b1a412e0fb1675ec53c95242/68747470733a2f2f62616467652e667572792e696f2f6a732f737070756c6c2e737667)](https://badge.fury.io/js/sppull) [![Downloads](https://camo.githubusercontent.com/1b0c429f74651e53c9fd0fa47ec808874855656e/68747470733a2f2f696d672e736869656c64732e696f2f6e706d2f646d2f737070756c6c2e737667)](https://www.npmjs.com/package/sppull) [![Build Status](https://camo.githubusercontent.com/1c64dc81898f1d36154349ed8b71f8c3bece0b16/68747470733a2f2f6b6f6c7479616b6f762e76697375616c73747564696f2e636f6d2f53504e6f64652f5f617069732f6275696c642f7374617475732f737070756c6c3f6272616e63684e616d653d6d6173746572)](https://camo.githubusercontent.com/1c64dc81898f1d36154349ed8b71f8c3bece0b16/68747470733a2f2f6b6f6c7479616b6f762e76697375616c73747564696f2e636f6d2f53504e6f64652f5f617069732f6275696c642f7374617475732f737070756c6c3f6272616e63684e616d653d6d6173746572) [![Gitter](https://camo.githubusercontent.com/20d7543bc8280bf8134b686c46c7b7e2c0a467fd/68747470733a2f2f6261646765732e6769747465722e696d2f67697474657248512f6769747465722e706e67)](https://gitter.im/sharepoint-node/Lobby)

Node.js module for downloading files from SharePoint document libraries.

### New in version 2.2.0

Files streaming download:

* Download files of any supported size
* Effective memmory consumption while fetching large files

### New in version 2.1.0

Performance in SPO and HTTPS environments is improved. Download on multiple objects is x2 faster now!

### New in version 2.0.6

Smart re-download mechanism. Existing files with no changes are ignored from the download.

### Supported SharePoint versions

* SharePoint Online
* SharePoint On-Prem (2019, 2016, 2013)

### How to use

#### Install

```
npm install sppull
```

#### Demo

![How it works](https://camo.githubusercontent.com/44597adb431b866999d742a0ebf50691a316e561/687474703a2f2f6b6f6c7479616b6f762e72752f696d616765732f737070756c6c2d64656d6f2e676966)

#### Usage

```typescript
var sppull = require("sppull").sppull;

var context = {/*...*/};
var options = {/*...*/};

sppull(context, options)
  .then(successHandler)
  .catch(errorHandler);
```

**Arguments**

**Context**

* `siteUrl` - SharePoint site (SPWeb) url \[string, required]
* `creds`
  * `username` - user name for SP authentication \[string, optional in case of some auth methods]
  * `password` - password \[string, optional in case of some auth methods] ...

**Additional authentication options:**

Since communication module (sp-request), which is used in sppull, had received additional SharePoint authentication methods, they are also supported in sppull.

* SharePoint On-Premise (Add-In permissions)
  * `clientId`
  * `issuerId`
  * `realm`
  * `rsaPrivateKeyPath`
  * `shaThumbprint`
* SharePoint On-Premise (NTLM handshake - more commonly used scenario):
  * `username` - username without domain
  * `password`
  * `domain` / `workstation`
* SharePoint Online (Add-In permissions):
  * `clientId`
  * `clientSecret`
* SharePoint Online (SAML based with credentials - more commonly used scenario):
  * `username` - user name for SP authentication \[string, required]
  * `password` - password \[string, required]
* ADFS user credantials:
  * `username`
  * `password`
  * `relyingParty`
  * `adfsUrl`

For more information please check node-sp-auth [credential options](https://github.com/s-KaiNet/node-sp-auth#params) and [wiki pages](https://github.com/s-KaiNet/node-sp-auth/wiki).

**Options**

* `spRootFolder` - root folder in SharePoint to pull from \[string, required]
* `dlRootFolder` - local root folder where files and folders will be saved to \[string, optional, default: `./Downloads`]
* `spBaseFolder` - base folder path which is omitted then saving files locally \[string, optional, default: equals to spRootFolder]
* `recursive` - to pull all files and folders recursively \[boolean, optional, default: `true`]
* `ignoreEmptyFolders` - to ignore local creation of SharePoint empty folders \[boolean, optional, default: `true`]
* `foderStructureOnly` - to ignore files, recreate only folders' structure \[boolean, optional, default: `false`]
* `strictObjects` - array of files and folders relative paths within the `spRootFolder` to proceed explicitly, \[array of strings, optional]
* `camlCondition` - SharePoint CAML conditions to use \[string, optional]
* `spDocLibUrl` - SharePoint document library URL \[string, mandatory with `camlCondition`]
* `metaFields` - array of internal field names to request along with the files \[array of strings, optional]
* `createEmptyFolders` - to create empty folders along with documents download task \[boolean, optional, default: `true`]
* `omitFolderPath` - folder path pattern which is omitted from final downloaded files path \[string, optional]
* `muteConsole` - to mute console messages during transport queries to SharePoint API \[boolean, optional, default: `false`]

**Overloads / cases**

* All files with folder structure from spRootFolder
* Files from spRootFolder folder, first hierarchy level only
* Folders structure from spRootFolder without files
* Files based on array of paths provided strictly \[works with array of files only right now]
* Files based on CAML query conditions
* Pull for documents metadata to use it in callback's custom logic

Use case scenarios can be found on the [Scenarios](https://github.com/koltyakov/sppull/tree/master/docs/Scenarios.md) page. This page suggests combinations of options which are optimal for certain use cases.

**successHandler**

Callback gets called upon successful files download.

**errorHandler**

Callback gets executed in case of exception inside `sppull`. Accepts error object as first argument for callback.

### Samples

Refer to the [Scenarios](https://github.com/koltyakov/sppull/tree/master/docs/Scenarios.md) page for suggested options combinations available with `sppull`.

#### Basic usage

```typescript
const { sppull } = require("sppull");

const context = {
  siteUrl: "http://contoso.sharepoint.com/subsite",
  creds: {
    username: "user@contoso.com",
    password: "_Password_"
  }
};

const options = {
  spRootFolder: "Shared%20Documents/Contracts",
  dlRootFolder: "./Downloads/Contracts"
};

/*
 * All files will be downloaded from http://contoso.sharepoint.com/subsite/Shared%20Documents/Contracts folder
 * to __dirname + /Downloads/Contracts folder.
 * Folders structure will remain original as it is in SharePoint's target folder.
*/
sppull(context, options)
  .then((downloadResults) => {
    console.log("Files are downloaded");
    console.log("For more, please check the results", JSON.stringify(downloadResults));
  })
  .catch((err) => {
    console.log("Core error has happened", err);
  });
```

#### Passwords storage

To eliminate any local password storing if preferable to use any two-way hashing technique, like [cpass](https://github.com/koltyakov/cpass).

### Inspiration and references

This project was inspired by [spsave](https://github.com/s-KaiNet/spsave) by [Sergei Sergeev](https://github.com/s-KaiNet) and [gulp-spsync](https://github.com/wictorwilen/gulp-spsync) by [Wictor Wilén](https://github.com/wictorwilen) projects.

SPPull depends heavily on [sp-request](https://github.com/s-KaiNet/sp-request) module and use it to send REST queries to SharePoint.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://node.spflow.com/packages/sppull.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.