quarta-feira, 13 de maio de 2020

DTSX\SSIS - importação de arquivos em lote de sql integration server


DTSX\SSIS - importação de arquivos em lote de sql integration server


Considerações 

1- Dentro do servidor crie uma pasta - NO meu caso criei em C:\Sistemas\importa_dtsx
2- Os scripts devem ser executados com usuário que seja sql admin, dentro da maquina
3- Todos os dtsx informados estão com senha de criptografia então faço o tratamento para isso na importação
4- Verifique o caminho onde esta o executável do DTSUTILL (este e o cara para automação)
5- Crie a pasta C:\Sistemas\import_dtsx\Importando\
6- Crie a pasta C:\sistemas\importados_dtsx\

Normalmente crio dois scripts um para backup e outro para importação.


BACKUP

CRIE UM ARQUIVO TIPO backup.bat BAT COM OS COMANDOS

-------------------------------------------------------------

REM Busca a data no formato AAAAMMDD para concatenar no nome do arquivo de log
set AAAAMMDD=
for /F "tokens=1-4 delims=/ " %%a in ('date /T') do set AAAAMMDD=%%d%%b%%c

 @echo off

MKDIR C:\Sistemas\BKPDTS\COMSENHA%AAAAMMDD% >logbkp%AAAAMMDD%.txt
MKDIR C:\Sistemas\BKPDTS\SEMSENHA%AAAAMMDD% >logbkp%AAAAMMDD%.txt
MKDIR c:\Sistemas\importa_dtsx\admin\

echo %date%-%time% - Iniciando a execucao dos scripts > logbkp%AAAAMMDD%.txt

for %%g in ( *.dtsx ) do (
echo %date%-%time% - importando o Dtsx: %%g >> logbkp%AAAAMMDD%.txt
echo %%~ng >>logbkp%AAAAMMDD%.txt
echo %date%-%time% - BKP do Dtsx: %%g >> logbkp%AAAAMMDD%.txt
"C:\SQLBIN\Microsoft SQL Server\130\DTS\Binn\DTUtil" /SQL "\%%~ng" /Decrypt SENHA /SourceS "Servidor\INSTANCIA" /Encrypt  FILE;"C:\Sistemas\BKPDTS\COMSENHA%AAAAMMDD%\%%g";3;SENHA  /Q >> bkpcomsenha.txt
"C:\SQLBIN\Microsoft SQL Server\130\DTS\Binn\DTUtil" /SQL "\%%~ng" /SourceS "Servidor\INSTANCIA" /Encrypt  FILE;"C:\Sistemas\BKPDTS\TRIBANCO\SEMSENHA%AAAAMMDD%\%%g";3;SENHA /Q >> bkpsemsenha.txt
  "echo %date%-%time% - bkp o Dtsx: %%~ng >> logbkp%AAAAMMDD%.txt
)
echo %date%-%time% - Fim dos bkps >> logbkp%AAAAMMDD%.txt
echo %date%-%time% - Execucao dos bkps finalizada, verifique o arquivo logbkp%AAAAMMDD%.txt.

move  logbkp%AAAAMMDD%.txt c:\Sistemas\importa_dtsx\admin\
move  copy.txt c:\Sistemas\importa_dtsx\admin\copy%AAAAMMDD%.txt

-----------------------------------------------

Observe que e feito 2 tipos de backup com senha e sem senha.
Ele executa com os arquivos SSIS na pasta ou seja tenho arquivos clientes.dtsx para importar, eu executo o bat ele vai extrair o backup do arquivo caso exista.


IMPORTAÇÃO DTSX

CRIE UM ARQUIVO TIPO BAT COM OS COMANDOS

REM Busca a data no formato AAAAMMDD para concatenar no nome do arquivo de log
set AAAAMMDD=
for /F "tokens=1-4 delims=/ " %%a in ('date /T') do set AAAAMMDD=%%d%%b%%

@echo off
cd "c:\Sistemas\import_dtsx\"
del "C:\Sistemas\import_dtsx\*.txt"
echo %date%-%time% - Iniciando processo > log.txt
echo %date%-%time% - apagando arquivos residuais >> log.txt
del "C:\Sistemas\import_dtsx\Importando\*.dtsx" >> log.txt
rem removendo arquivos já importados >> log.txt
del "C:\sistemas\importados_dtsx\*dtsx" >>log.txt
rem Inciando processamento >> log.txt
echo inico do backup -log.txt
call backup.bat
echo fim do backup -log.txt
echo inico da importação -log.txt
for %%g in ( *.dtsx ) do (
echo %date%-%time% - importando o Dtsx: %%g >> log.txt
echo %%~ng >>log.txt
echo IMPORTANDO ARQUIVO %%g >>log.txt
"C:\SQLBIN\Microsoft SQL Server\130\DTS\Binn\DTUtil" /File "C:\Sistemas\import_dtsx\%%g" /Decrypt SENHA /DestS "Servidor\INSTANCIA" /COPY sql;"%%~ng" /Q >> importfase1.txt
"C:\SQLBIN\Microsoft SQL Server\130\DTS\Binn\DTUtil" /SQL "\%%~ng" /Decrypt SENHA /SourceS "Servidor\INSTANCIA" /Encrypt  FILE;"C:\Sistemas\import_dtsx\Importando\%%g";0 /Q >> importfase2.txt
"C:\SQLBIN\Microsoft SQL Server\130\DTS\Binn\DTUtil" /File "C:\Sistemas\import_dtsx\%%g" /DestS "Servidor\INSTANCIA" /COPY sql;"%%~ng" /Q >> log.txt
  echo %date%-%time% - movendo arquivo %%g para importados >> log.txt
  move "C:\Sistemas\import_dtsx\%%g" "C:\sistemas\importados_dtsx\" >>log.txt
)
echo %date%-%time% - Fim da importacao >> log.txt
echo %date%-%time% - Execucao dos scripts finalizada, verifique o arquivo log.txt.
del C:\Sistemas\importados_dtsx\*.dtsx
copy "C:\Sistemas\import_dtsx\log.txt" "C:\Sistemas\import_dtsx\admin\log%AAAAMMDD%.txt"


Observe que e gerado um log.txt  e um log por fase de importação importfase2.txt e importfase1.txt, e após importação e movido para C:\sistemas\importados_dtsx\.



Espero que isso ajude 

segunda-feira, 30 de março de 2015

Verifica ultima data de backup de todos os databases da instância

Este comando verifica a ultima data de cada database da instância, bem como que foi o executor
---------------------------------------------------
SELECT  
  T1.Name As ab_taxamedia
 ,Coalesce(Convert(varchar(12), MAX(T2.Backup_Finish_Date), 101),'Not Yet Taken') As LastBackUpTaken
 ,Coalesce(Convert(varchar(12), MAX(T2.User_Name), 101), 'NA') As UserName
FROM
   sys.sysdatabases T1
LEFT OUTER JOIN
  msdb.dbo.backupset T2
ON
   T2.Database_Name = T1.Name
GROUP BY
    T1.Name
ORDER BY
    T1.Name

 

Verifica versão do dtsx SQL SERVER

Segue comandos para averiguar a versão publicada de SSIS (.dtsx)
OBS.: Substitui o PackageName pelo nome do pacote.

--------------------------------------------
Verifica  versão do dtsx - 2005
--------------------------------------------

SELECT PACK.name AS PackageName
      ,PACK.isencrypted AS IsEncrypted
      ,PACK.createdate AS CreateDate
      ,CONVERT(varchar(10), vermajor)
       + '.' + CONVERT(varchar(10), verminor)
       + '.' + CONVERT(varchar(10), verbuild) AS Version
    ,DATALENGTH(PACK.packagedata) AS PackageSize
FROM msdb.dbo.sysdtspackages90 AS PACK
where PACK.name in ('PackageName')
order by CreateDate desc

-----------------------------------------
Verifica  versão do dtsx - 2008
-----------------------------------------

SELECT PACK.name AS PackageName
      ,PACK.isencrypted AS IsEncrypted
      ,PACK.createdate AS CreateDate
      ,CONVERT(varchar(10), vermajor)
       + '.' + CONVERT(varchar(10), verminor)
       + '.' + CONVERT(varchar(10), verbuild) AS Version
    ,DATALENGTH(PACK.packagedata) AS PackageSize
FROM msdb.dbo.sysssispackages AS PACK
where PACK.name in ('PackageName')
order by CreateDate desc

Verifica informações de uma instância SQL Server

Verifica, informações de uma instância SQL SERVER, tais como
     Versão do sql server
     Versão do S.O
     Data e hora da Instância
---------------------------------------------------------------------
use database
go
SET NOCOUNT ON
PRINT @@Version
PRINT DB_NAME()
PRINT GETDATE()
GO
----------------------------------------------------------------------

Comando para averiguar tamanho (utilizado e disponível) dos discos utilizados pela instância. SQL Server 2008

Somente executar a query abaixo para averiguar o tamanho dos discos utilizados pela instância.


SELECT DISTINCT
VS.volume_mount_point [Montagem]
,VS.logical_volume_name AS [Volume]
,CAST(CAST(VS.total_bytes AS DECIMAL(19,2))/1024 /1024 /1024 AS DECIMAL (10,2)) AS [Total (GB)]
,CAST(CAST(VS.available_bytes AS DECIMAL(19,2))/1024 /1024 /1024 AS DECIMAL (10,2)) AS [Espaço Disponível (GB)]
,CAST((CAST(VS.available_bytes AS DECIMAL(19,2)) / CAST(VS.total_bytes AS DECIMAL(19,2)) * 100 ) AS DECIMAL(10,2)) AS [Espaço Disponível ( % )],CAST((100 - CAST(VS.available_bytes AS DECIMAL(19,2)) / CAST(VS.total_bytes AS DECIMAL(19,2)) * 100) AS DECIMAL (10,2)) AS [Espaço em uso ( % )]
FROM sys.master_files AS MF
CROSS APPLY [sys].[dm_os_volume_stats](MF.database_id, MF.FILE_ID) AS VS
WHERE CAST(VS.available_bytes AS DECIMAL(19,2)) / CAST(VS.total_bytes AS DECIMAL(19,2)) * 100 < 100


OBS.:  Este comando é somente para Sql Server 2008

Verificando data e hora da instancia SQL server

Comando para averiguar a data e hora de uma determinada instancia do SQL Server

use tempdb
go
declare @DATA as datetime = GETDATE()
select @DATA
go
 

Utilização de CPU da instancia SQLServer 2008/2005

Utilize a seguinte linha de comando para averiguar a utilização de cpu para a devida instância.


DECLARE @ts_now BIGINT
 SELECT @ts_now = ms_ticks FROM sys.dm_os_sys_info

 SELECT top 1 record_id,
  DATEADD(MS, -1 * (@ts_now - [timestamp]), GETDATE()) AS EventTime,
  SQLProcessUtilization,
  SystemIdle,
  100 - SystemIdle - SQLProcessUtilization AS OtherProcessUtilization
 FROM (
  SELECT
   record.value('(./Record/@id)[1]', 'int') AS record_id,
   record.value('(./Record/SchedulerMonitorEvent/SystemHealth/SystemIdle)[1]', 'int') AS SystemIdle,
   record.value('(./Record/SchedulerMonitorEvent/SystemHealth/ProcessUtilization)[1]', 'int') AS SQLProcessUtilization,
   TIMESTAMP
  FROM (
   SELECT TIMESTAMP, CONVERT(XML, record) AS record
   FROM sys.dm_os_ring_buffers
   WHERE ring_buffer_type = N'RING_BUFFER_SCHEDULER_MONITOR'
   AND record LIKE '% %') AS x
  ) AS y
 ORDER BY record_id DESC
Go

quarta-feira, 11 de fevereiro de 2015

Verificando a versão do DTSX do Sql server importado



Verifica  versão do dtsx - 2005

SELECT PACK.name AS PackageName
      ,PACK.isencrypted AS IsEncrypted
      ,PACK.createdate AS CreateDate
      ,CONVERT(varchar(10), vermajor)
       + '.' + CONVERT(varchar(10), verminor)
       + '.' + CONVERT(varchar(10), verbuild) AS Version
    ,DATALENGTH(PACK.packagedata) AS PackageSize
FROM msdb.dbo.sysdtspackages90 AS PACK
where PACK.name in ('PackageName')
order by CreateDate desc


Verifica  versão do dtsx - 2008

SELECT PACK.name AS PackageName
      ,PACK.isencrypted AS IsEncrypted
      ,PACK.createdate AS CreateDate
      ,CONVERT(varchar(10), vermajor)
       + '.' + CONVERT(varchar(10), verminor)
       + '.' + CONVERT(varchar(10), verbuild) AS Version
    ,DATALENGTH(PACK.packagedata) AS PackageSize
FROM msdb.dbo.sysssispackages AS PACK
where PACK.name in ('PLDAdviceCargaMovFinanceiro')
order by CreateDate desc

segunda-feira, 29 de setembro de 2014


Trust assemby







O  Trust assembly é necessário para assinar executáveis e dlls que serão executadas em locais diferentes ao servidor de origem.

OBS.: Arquivos .config  .xml .txt .log  não é necessário trust

Exemplo executável esta no \\tribanco.com.br\hmldfs\ e será executada na trbsvshml01

Framework por versão do visual Studio:

Generation
Version number
Release date
Development tool
Distributed with
1.0.3705.0
2002-02-13
N/A
1.1.4322.573
2003-04-24
2.0.50727.42
2005-11-07
3.0.4506.30
2006-11-06
3.5.21022.8
2007-11-19
4.0.30319.1
2010-04-12
N/A
4.5.50709.17929
2012-08-15

 Como funcionam as versões de trust por generation do visual Studio:

Versão do trust assembly
Generation do viual studio
1.1
1.0 – 1.1
2.0
2.0 – 3.0 – 3.5
Não é necessário trust
4.0 – 4.5

 

segunda-feira, 22 de setembro de 2014

Problema de compatibilidade windows 8 e Cisco Vpn

Após instalar o windows 8 na minha maquina tive um problema em conectar na vpn que indentifique nos logs:

1      12:54:15.440  06/24/12  Sev=Warning/3    CVPND/0xE340000C
The Client was unable to enable the Virtual Adapter because it could not open the device.
2      12:54:15.455  06/24/12  Sev=Warning/3    CVPND/0xE340000C
The Client was unable to enable the Virtual Adapter because it could not open the device.
3      12:54:15.455  06/24/12  Sev=Warning/2    IKE/0xE300009B
Failed to active IPSec SA: Unable to enable Virtual Adapter (NavigatorQM:936)
4      12:54:15.455  06/24/12  Sev=Warning/2    IKE/0xE30000A7
Unexpected SW error occurred while processing Quick Mode negotiator:(Navigator:2263)

Onde a solução foi

Abrir o regedit e va até : HKLM\SYSTEM\CurrentControlSet\Services\CVirtA
Feito isso verifique os casos abaixo:

Se x86, altere o valor da seguinte forma "@oem8.inf,%CVirtA_Desc%;Cisco Systems VPN Adapter" para "Cisco Systems VPN Adapter"

Se x64, altere o valor da seguinte forma "@oem8.inf,%CVirtA_Desc%;Cisco Systems VPN Adapter for 64-bit Windows" para "Cisco Systems VPN Adapter for 64-bit Windows"

Após isto Funcionou normalmente


quinta-feira, 28 de agosto de 2014

Backup \ Restore - passo a passo por linha de comando no SQL SERVER


 Backup por linha de comando

BACKUP DATABASE DB_BANCO TO DISK=''Caminhoparasalvar\nomedobanco_yyyymmdd.bak '
GO
 ---------------------------------------------------------------------------------------------------------------------------

Robocopy – caso necessário copiar o database para outro servidor

D:\Robocopy\robocopy.exe \\caminhodeorigem\ \\caminhodedestino\ /TEE /LOG+:diretoriodolog\nomedolog.log /COPYALL
---------------------------------------------------------------------------------------------------------------------------

 Restore por linha do comando

RESTORE DATABASE DB_BANCO FROM DISK = 'caminhoondeestasalvo\arquivodobackup.bak'
WITH REPLACE,
MOVE 'db_banco _log' TO 'diretorioondeestasalvodoarquivodelogparasobrescrever\nomedoarquivodelog.ldf',
MOVE 'db_banco' TO 'diretorioondeestasalvodoarquivomdfparasobrescrever\nomedoarquivodemdf.mdf'
GO
---------------------------------------------------------------------------------

-- Verificando ultima data de restore

declare @database varchar(50)
set @database = 'SEU_DATABASE'
SELECT rsh.destination_database_name AS [Database]
,rsh.user_name AS [Restored By]
,rsh.restore_date AS [Restore Started]
,bmf.physical_device_name AS [Restored From]
,rf.destination_phys_name AS [Restored To]
FROM msdb.dbo.restorehistory rsh
INNER JOIN msdb.dbo.backupset bs
ON rsh.backup_set_id = bs.backup_set_id
INNER JOIN msdb.dbo.restorefile rf
ON rsh.restore_history_id = rf.restore_history_id
INNER JOIN msdb.dbo.backupmediafamily bmf
ON bmf.media_set_id = bs.media_set_id
where rsh.destination_database_name = @database
and rsh.restore_type = 'D'
ORDER BY rsh.restore_date DESC

 -----------------------------------------------------------------------------------

 Gera sincronismo usuários
use DB_BANCO
go
set nocount on
select 'use '+db_name()+char(10)+'go'
select 'set nocount on'
select 'select '''+name+''''+char(10)+
       'exec sp_change_users_login @Action = ''Auto_Fix'',@UserNamePattern = '''+name+''''+char(10)+'go'
from sys.sysusers where uid > 4
and issqluser = 1
and sid not in (select sid from master..syslogins)
go
------------------------------------------------------------------------------ 

Dicas para quem trabalha com index no SQL Server


Tabelas que não contêm índices clusteriazados.

 select distinct(tb.name) as Table_name, p.rows from sys.objects tb join sys.partitions p on p.object_id = tb.object_id Where type = ‘U’ and tb.object_id not in ( select ix.object_id from sys.indexes ix where type = 1 ) order by p.rows desc

------------------------------------------------------------------------------ 
  

Tabelas que não possuem chave primária.
 select distinct(tb.name) as Table_name, p.rows from sys.objects tb join sys.partitions p on p.object_id = tb.object_id Where type = ‘U’ and tb.object_id not in ( select ix.parent_object_id from sys.key_constraints ix where type = ‘PK’ ) order by p.rows desc

------------------------------------------------------------------------------ 
 

Tabelas que mais seriam beneficiadas com novos índices.
SELECT TOP 15 AVG((avg_total_user_cost * avg_user_impact * (user_seeks + user_scans))) as Impacto,mid.object_id, mid.statement as Tabela FROM sys.dm_db_missing_index_group_stats AS migs JOIN sys.dm_db_missing_index_groups AS mig ON migs.group_handle = mig.index_group_handle JOIN sys.dm_db_missing_index_details AS mid ON mig.index_handle = mid.index_handle and database_id = db_id(‘SEU Banco de Dados AQUI ‘) GROUP BY mid.object_id, mid.statement ORDER BY Impacto DESC;

 ------------------------------------------------------------------------------ 
 

Top 15 índices, sugeridos pelo SGBD.
SELECT TOP 15 (avg_total_user_cost * avg_user_impact * (user_seeks + user_scans)) as Impacto, migs.group_handle, mid.index_handle, migs.user_seeks,migs.user_scans, mid.object_id, mid.statement, mid.equality_columns, mid.inequality_columns, mid.included_columns FROM sys.dm_db_missing_index_group_stats AS migs JOIN sys.dm_db_missing_index_groups AS mig ON migs.group_handle = mig.index_group_handle JOIN sys.dm_db_missing_index_details AS mid ON mig.index_handle = mid.index_handle and database_id = db_id(‘SEU Banco de Dados AQUI’) –and mid.object_id = object_id(‘tabela’) — se desejar ver apenas para uma tabela específica order by Impacto DESC;

 ------------------------------------------------------------------------------ 

 

Índices nunca utilizados pelo SGBD.
select tb.name as Table_Name, ix.name as Index_Name, ix.type_desc, leaf_insert_count,leaf_delete_count, leaf_update_count, nonleaf_insert_count ,nonleaf_delete_count, nonleaf_update_count from sys.dm_db_index_usage_stats vw join sys.objects tb on tb.object_id = vw.object_id join sys.indexes ix on ix.index_id = vw.index_id and ix.object_id = tb.object_id join sys.dm_db_index_operational_stats(db_id(‘AdventureWorks’), Null, NULL, NULL) vwx on vwx.object_id = tb.object_id and vwx.index_id = ix.index_id where vw.database_id = db_id(‘AdventureWorks’)and vw.user_seeks = 0 and vw.user_scans = 0 and vw.user_lookups = 0 and vw.system_seeks = 0 and vw.system_scans = 0 and vw.system_lookups = 0 Order By leaf_insert_count desc, tb.name asc, ix.name asc

 ------------------------------------------------------------------------------ 

 

2 - indices de sistemas nunca utilizados
select
tb.name as Table_Name,
ix.name as Index_Name,
ix.type_desc,
leaf_insert_count,
leaf_delete_count,
leaf_update_count,
nonleaf_insert_count,
nonleaf_delete_count,
nonleaf_update_count
from sys.dm_db_index_usage_stats vw
join sys.objects tb on tb.object_id = vw.object_id
join sys.indexes ix on ix.index_id = vw.index_id and
 ix.object_id = tb.object_id
join sys.dm_db_index_operational_stats(db_id('msdb_org'), Null, NULL, NULL)
vwx on vwx.object_id = tb.object_id and vwx.index_id = ix.index_id
where ix.name <> 'clust' and ix.name <> 'nc' and ix.name <> 'nc1' and ix.name <> 'nc2' and ix.name <> 'nc3'  and ix.name <> 'clst'
And vw.user_seeks = 0 and vw.user_scans = 0 and vw.user_lookups = 0 
Order By leaf_insert_count desc, tb.name asc, ix.name asc
------------------------------------------------------------------------------



Avaliando índices.
select ix.name, ix.type_desc, vwy.partition_number, vw.user_seeks, vw.last_user_seek, vw.user_scans, vw.last_user_scan, vw.user_lookups, vw.user_updates as ‘Total_User_Escrita’,(vw.user_scans + vw.user_seeks + vw.user_lookups) as ‘Total_User_Leitura’,vw.user_updates – (vw.user_scans + vw.user_seeks + vw.user_lookups) as ‘Dif_Read_Write’,ix.allow_row_locks, vwx.row_lock_count, row_lock_wait_count, row_lock_wait_in_ms,ix.allow_page_locks, vwx.page_lock_count, page_lock_wait_count, page_lock_wait_in_ms,ix.fill_factor, ix.is_padded, vwy.avg_fragmentation_in_percent, vwy.avg_page_space_used_in_percent, ps.in_row_used_page_count as Total_Pagina_Usada,ps.in_row_reserved_page_count as Total_Pagina_Reservada,convert(real,ps.in_row_used_page_count) * 8192 / 1024 / 1024 as Total_Indice_Usado_MB, convert(real,ps.in_row_reserved_page_count) * 8192 / 1024 / 1024 as Total_Indice_Reservado_MB,page_io_latch_wait_count, page_io_latch_wait_in_ms from sys.dm_db_index_usage_stats vw join sys.indexes ix on ix.index_id = vw.index_id and ix.object_id = vw.object_id join sys.dm_db_index_operational_stats(db_id(‘ArtigosMS’), OBJECT_ID(N’Log’), NULL, NULL) vwx on vwx.index_id = ix.index_id and ix.object_id = vwx.object_id join sys.dm_db_index_physical_stats(db_id(‘ArtigosMS’), OBJECT_ID(N’Log’), NULL, NULL , ‘SAMPLED’) vwy on vwy.index_id = ix.index_id and ix.object_id = vwy.object_id and vwy.partition_number = vwx.partition_number join sys.dm_db_partition_stats PS on ps.index_id = vw.index_id and ps.object_id = vw.object_id where vw.database_id = db_id(‘ArtigosMS’) AND object_name(vw.object_id) = ‘Log’ order by user_seeks desc, user_scans desc

 ------------------------------------------------------------------------------ 


Tabelas com maior quantidade de índices.
 select x.id, x.table_name, x.Total_index, count(*) as Total_column from sys.columns cl join (select ix.object_id as id, tb.name as table_name, count(ix.object_id) as Total_index from sys.indexes ix join sys.objects tb on tb.object_id = ix.object_id and tb.type = ‘u’group by ix.object_id, tb.name) x on x.id = cl.object_id group by id, table_name, Total_index order by 3 desc